py.bench -- py.test based benchmarking
The idea is to reuse py.test infrastructure to
o collect tests
o run collected tests
o present test runs results
and substitute 'test' -> 'benchmark'.
What we do is
1. collect benchmark files that start with 'bench_' prefix
2. execute two kind of benchmarks
a. 'bench_<smth>', and
b. 'timeit_<smth>',
for bennch_XXX we simple mesure time to execute, and for timeit_XXX in
essence we do IPython's %timeit on it.
For actually running benchmarks small py.test like utility is provided
and installed for the benefit of all.
Signed-off-by: Kirill Smelkov <kirr@landau.phys.spbu.ru>
Signed-off-by: Ondrej Certik <ondrej@certik.cz>