Initial SymPy benchmark suite
I've just went through our commit history and searched for commits with
%timeit or %time in their description and extracted benchmarks from
there.
So now we have at least something. Here is how it looks like:
kirr@evo:~/src/sympy/sympy-git$ ./bin/py.bench sympy/
====================== test process starts =======================
executable: /usr/bin/python (2.5.2-final-0)
using py lib: /home/kirr/src/tools/py/py/py <rev unknown>
sympy/benchmarks/bench_symbench.py[8] ........
sympy/core/benchmarks/bench_arit.py[10] ..........
sympy/core/benchmarks/bench_assumptions.py[2] ..
sympy/core/benchmarks/bench_basic.py[4] ....
sympy/core/benchmarks/bench_expand.py[4] ....
sympy/core/benchmarks/bench_numbers.py[21] .....................
sympy/core/benchmarks/bench_sympify.py[2] ..
sympy/functions/elementary/benchmarks/bench_exp.py[1] .
sympy/functions/special/benchmarks/bench_special.py[1] .
sympy/integrals/benchmarks/bench_integrate.py[4] ....
sympy/integrals/benchmarks/bench_trigintegrate.py[2] ..
sympy/matrices/benchmarks/bench_matrix.py[4] ....
sympy/polynomials/benchmarks/bench_factor.py[2] ..
sympy/series/benchmarks/bench_limit.py[1] .
sympy/solvers/benchmarks/bench_solvers.py[1] .
=========== tests finished: 68 passed in 64.66 seconds ===========
==============================
*** BENCHMARKING RESULTS ***
==============================
bench_R1 | 670 ms | real(f(f(f(f(f(f(f(f(f(f(i/2)))))))))))
bench_R2 | 664 ms | Hermite polynomial hermite(15, y)
bench_R3 | 731 us | a = [bool(f==f) for _ in range(10)]
bench_R4 | 6.2 us | # we don't have Tuples
bench_R5 | 12.9 ms | blowup(L, 8); L=uniq(L)
bench_R6 | 465 ms | sum(trim((x+sin(i))/x+(x-sin(i))/x) for i in xrange(100))
bench_R10 | 150 ms | v = [-pi,-pi+1/10..,pi]
bench_R11 | 3.17 s | a = [random() + random()*I for w in [0..1000]]
timeit_neg | 20.5 us | -x
timeit_Add_x1 | 28.7 us | x+1
timeit_Add_1x | 42.3 us | 1+x
timeit_Add_x05 | 24.3 us | x+0.5
timeit_Add_xy | 24.3 us | x+y
timeit_Add_xyz | 23.3 us | Add(*[x,y,z])
timeit_Mul_xy | 24.1 us | x*y
timeit_Mul_xyz | 22.8 us | Mul(*[x,y,z])
timeit_Div_xy | 39.4 us | x/y
timeit_Div_2y | 46.6 us | 2/y
timeit_x_is_integer | 4.72 us | x.is_integer
timeit_Integer_is_irrational | 1.06 us | i3.is_irrational
timeit_Symbol_meth_lookup | 1.07 us | x.diff # no call, just method lookup
timeit_S_lookup | 765 ns | S.Exp1
timeit_C_lookup | 627 ns | C.Add
timeit_Symbol_eq_xy | 11.3 us | x == y
timeit_expand_nothing_todo | 363 us | p.expand()
bench_expand_32 | 5.66 s | (x+y+z+1)**32 -> expand
timeit_expand_complex_number_1 | 3.6 ms | ((2+3*I)**1000).expand(complex=True)
timeit_expand_complex_number_2 | 4.94 ms | ((2+3*I/4)**1000).expand(complex=True)
timeit_Integer_create | 3.42 us | Integer(2)
timeit_Integer_int | 4.96 us | int(i3)
timeit_neg_one | 2.51 us | -S.One
timeit_Integer_neg | 6.72 us | -i3
timeit_Integer_abs | 4.21 us | abs(i3)
timeit_Integer_sub | 13 us | i3 - i3
timeit_abs_pi | 2.86 us | abs(pi)
timeit_neg_oo | 2.18 us | -oo
timeit_Integer_add_i1 | 10.3 us | i3+1
timeit_Integer_add_ij | 13 us | i3+i4
timeit_Integer_add_Rational | 45.3 us | i3+r34
timeit_Integer_mul_i4 | 10.5 us | i3*4
timeit_Integer_mul_ij | 13.2 us | i3*i4
timeit_Integer_mul_Rational | 42.7 us | i3*r34
timeit_Integer_eq_i3 | 4.56 us | i3==3
timeit_Integer_ed_Rational | 33.8 us | i3==r34
timeit_integer_nthroot | 18.7 us | integer_nthroot(100, 2)
timeit_number_igcd_23_17 | 26.7 us | igcd(23,17)
timeit_number_igcd_60_3600 | 24.6 us | igcd(60,3600)
timeit_Rational_add_r1 | 40.8 us | r34+1
timeit_Rational_add_rq | 36.2 us | r34+q45
timeit_sympify_1 | 12.5 us | sympify(1)
timeit_sympify_x | 3.39 us | sympify(x)
timeit_exp_subs | 22 us | e.subs(q, y)
timeit_Ylm_xy | 766 us | Ylm(1,1, x,y)
bench_integrate_sin | 16 ms | integrate(sin(x), x)
bench_integrate_x1sin | 887 ms | integrate(x**1*sin(x), x)
bench_integrate_x2sin | 1.61 s | integrate(x**2*sin(x), x)
bench_integrate_x3sin | 3.04 s | integrate(x**3*sin(x), x)
timeit_trigintegrate_sin3x | 3.36 ms | trigintegrate(sin(x)**3, x)
timeit_trigintegrate_x2 | 610 us | trigintegrate(x**2, x) # -> None
timeit_Matrix__getitem_ii | 16.2 us | M[3,3]
timeit_Matrix__getitem_II | 18 us | M[i3,i3]
timeit_Matrix__getslice | 41.2 ms | M[:,:]
timeit_Matrix_zeronm | 39.6 ms | zeros((100, 100))
timeit_factor_x4_x_1 | 62.4 ms | factor(x**4+x+1)
bench_factor_x56_1 | 8.91 s | factor(x**56-1)
timeit_limit_1x | 175 us | limit(1/x, x, oo)
timeit_linsolve_trivial | 8.22 ms | solve_linear_system(M, *S)
Not ideal, but still worth to have ...
----
The next step will be to implement
a. storing benchmark data to files, and
b. comparing results of benchmarks runs.
>>> Anyone want to help with this?
I'm afraid I'll be working hard at work for the whole next week and
don't have time for this.
Signed-off-by: Kirill Smelkov <kirr@landau.phys.spbu.ru>
Signed-off-by: Ondrej Certik <ondrej@certik.cz>
14 files changed: