Solving the confusion generated by too many ways to run unittest in python

Question

I am trying to implement a full and clean way of testing python packages, one that would suit the folowing requirements:

• execute tests on clean machines without setting them up (virtualenv)
• gather results from multiple platforms
• gather results from multiple python interpreters
• code coverage (merge results from the multiple execution into a single report)
• be able to generate xml code coverage report so the build system can track how this change over time.
• be able to enable disable tests based on platform
• run several commands before the tests, like pep8 or autopep8
• run tests in parallel.

I used several approaches: nose + pytest, tox + pytest but recently discovered that pytest should be able to do most of the stuff.

The main problem is that I wasn't able to find a clear comparision regarding when it would be better to use one approach or another.

Can someone explain these and give some use cases or limitations of these configurations? ... just to make it clear when when you go for one approach or another.

In the end I do want to have these options:

• quicktest - run the tests locally, a must before each commmit
• fulltest - full tests, running them across all available platforms, a must before making a new release

Answer 1

py.test covers all your needs perfectly it's pros comparing to other test runners like nose:

• fixtures with dependency injection - eliminate the need of complicated oop for test setup
• simple but powerful plugin system with lots of useful plugins - plugins like pytest-xdist allow you to test on all platforms as you need, including windows, pytest-cov does the coverage, pytest-cache helps to run only last failed tests, etc
• parametrization allows you to use write-once run-multiple approach for tests using declarative parameters
• use of the simple assert statement vs complicated java-like syntax .assertEquals (http://pytest.org/latest/assert.html)
• more and more