How to run Tox with Travis-CI

Question

How do you test different Python versions with Tox from within Travis-CI?

I have a tox.ini:

[tox] envlist = py{27,33,34,35} recreate = True  [testenv] basepython =     py27: python2.7     py33: python3.3     py34: python3.4     py35: python3.5 deps =     -r{toxinidir}/pip-requirements.txt     -r{toxinidir}/pip-requirements-test.txt commands = py.test 

which runs my Python unittests in several Python versions and works perfectly.

I want to setup a build in Travis-CI to automatically run this when I push changes to Github, so I have a .travis.yml:

language: python python: -   "2.7" -   "3.3" -   "3.4" -   "3.5" install: -   pip install tox script: -   tox 

This technically seems to work, but it redundantly runs all my tests in each version of Python...from each version of Python. So a build that takes 5 minutes now takes 45 minutes.

I tried removing the python list from my yaml file, so Travis will only run a single Python instance, but that causes my Python3.5 tests to fail because the 3.5 interpreter can't be found. Apparently, that's a known limitation as Travis-CI won't install Python3.5 unless you specify that exact version in your config...but it doesn't do that for the other versions.

Is there a way I can workaround this?

Answer 1

For this I would consider using tox-travis. This is a plugin which allows use of Travis CI’s multiple python versions and Tox’s full configurability. To do this you will configure the .travis.yml file to test with Python:

sudo: false language: python python:     - "2.7"     - "3.4" install: pip install tox-travis script: tox 

This will run the appropriate testenvs, which are any declared env with py27 or py34 as factors of the name by default. Py27 or py34 will be used as fallback if no environments match the given factor.

Further Reading

Answer 2

For more control and flexibility you can manually define your matrix so that the Python version and tox environment match up:

language: python matrix:   include:     - python: 2.7       env: TOXENV=py27     - python: 3.3       env: TOXENV=py33     - python: 3.4       env: TOXENV=py34     - python: 3.5       env: TOXENV=py35     - python: pypy       env: TOXENV=pypy     - env: TOXENV=flake8 install:   - pip install tox script:   - tox 

In case it's not obvious, each entry in the matrix starts on a line which begins with a hyphen (-). Any items following that line which are indented are additional lines for that single item.

For example, all entries except for the last, are two lines. the last entry is only one line and does not contain a python setting; therefore, it simply uses the default Python version (Python 2.7 according to the Travis documentation). Of course, a specific Python version is not as important for that test. If you wanted to run such a test against both Python 2 and 3 (once each), then it is recommended to use the versions Travis installs by default (2.7 and 3.4) so that the tests complete more quickly as they don't need to install a non-standard Python version first. For example:

- python: 2.7   env: TOXENV=flake8 - python: 3.4   env: TOXENV=flake8 

The same works with pypy (second to last entry on matrix) and pypy3 (not shown) in addition to Python versions 2.5-3.6.

While the various other answers provide shortcuts which give you this result in the end, sometimes its helpful to define the matrix manually. Then you can define specific things for individual environments within the matrix. For example, you can define dependencies for only a single environment and avoid the wasted time installing that dependency in every environment.

- python: 3.5   env: TOXENV=py35 - env: TOXENV=checkspelling   before_install: install_spellchecker.sh - env: TOXENV=flake8 

In the above matrix, the install_spellchecker.sh script is only run for the relevant environment, but not the others. The before_install setting was used (rather than install), as using the install setting would have overridden the global install setting. However, if that's what you want (to override/replace a global setting), simply redefine it in the matrix entry. No doubt, various other settings could be defined for individual environments within the matrix as well.

Manually defining the matrix can provide a lot of flexibility. However, if you don't need the added flexibility, one of the various shortcuts in the other answers will keep your config file simpler and easier to read and edit later on.