How to run py.test and linters in `python setup.py test`

Question

I have a project with a setup.py file. I use pytest as the testing framework and I also run various linters on my code (pep8, pylint, pydocstyle, pyflakes, etc). I use tox to run these in several Python versions, as well as build documentation using Sphinx.

I would like to run my test suites as well as the linters on my source code with the python setup.py test task. If I acheive this, I will then just use python setup.py test as the command for running tests in my tox.ini file.

So my questions are:

1. Is it reasonable / good practice to do these actions with python setup.py test? Or should I just use some other tool for that, like writing those command directly in tox?
2. How do I get setup.py to do these actions in the test task?

I know py.test has integration instructions for setup.py test (here: http://pytest.org/latest/goodpractices.html#integrating-with-setuptools-python-setup-py-test-pytest-runner), but I'm looking for a more "arbitrary CLI commands" route, since I want to run several tools.

Answer 1

1. I personally prefer tox for these tasks, because

• it will also check if your deps install properly (on all py-versions)
• it can test your code with multiple python-versions (virtualenv)

And with your setup (since you're already using tox) i don't really see the benefits of writing python setup.py test instead of the exact test-commands into your tox.ini, because it will just add some more complexitiy (new users/contributors have to search two files (tox.ini and setup.py) for tests/tools running instead of one (tox.ini).

---

2.

To use this command, your project’s tests must be wrapped in a unittest test suite by either a function, a TestCase class or method, or a module or package containing TestCase classes.

setuptools#test