Python benchmark tool like nosetests?

Question

What I want

I would like to create a set of benchmarks for my Python project. I would like to see the performance of these benchmarks change as I introduce new code. I would like to do this in the same way that I test Python, by running the utility command like nosetests and getting a nicely formatted readout.

What I like about nosetests

The nosetests tool works by searching through my directory structure for any functions named test_foo.py and runs all functions test_bar() contained within. It runs all of those functions and prints out whether or not they raised an exception.

I'd like something similar that searched for all files bench_foo.py and ran all contained functions bench_bar() and reported their runtimes.

Questions

Does such a tool exist?

If not what are some good starting points? Is some of the nose source appropriate for this?

Answer 1

nosetests can run any type of test, so you can decide if they test functionality, input/output validity etc., or performance or profiling (or anything else you'd like). The Python Profiler is a great tool, and it comes with your Python installation.

import unittest
import cProfile

class ProfileTest(unittest.TestCase):
    test_run_profiler:
        cProfile.run('foo(bar)')
        cProfile.run('baz(bar)')

You just add a line to the test, or add a test to the test case for all the calls you want to profile, and your main source is not polluted with test code.

If you only want to time execution and not all the profiling information, timeit is another useful tool.