Tool (or combination of tools) for reproducible environments in Python

Question

I used to be a java developer and we used tools like ant or maven to manage our development/testing/UAT environments in a standardized way. This allowed us to handle library dependencies, setting OS variables, compiling, deploying, running unit tests, and all the required tasks. Also, the scripts generated guaranteed that all the environments were almost equally configured, and all the task were performed in the same way by all the members of the team.

I'm starting to work in Python now and I'd like your advice in which tools should I use to accomplish the same as described for java.

Answer 1

1. virtualenv to create a contained virtual environment (prevent different versions of Python or Python packages from stomping on each other). There is increasing buzz from people moving to this tool. The author is the same as the older working-env.py mentioned by Aaron.

2. pip to install packages inside a virtualenv. The traditional is easy_install as answered by S. Lott, but pip works better with virtualenv. easy_install still has features not found in pip though.

3. scons as a build tool, although you won't need this if you stay purely Python.

4. Fabric paste, or paver for deployment.

5. buildbot for continuous integration.

6. Bazaar, mercurial, or git for version control.

7. Nose as an extension for unit testing.

8. PyFit for FIT testing.

Answer 2

I also work with both java and python. For python development the maven equivalent is setuptools (http://peak.telecommunity.com/DevCenter/setuptools). For web application development I use this in combination with paster (http://pythonpaste.org/) for the deployment process

Answer 3

Other than easy_install?

For our Linux servers, we use easy_install and yum.

For our Windows development laptops, we use easy_install and a few MSI's for some projects.

Most of the Python libraries we use are source-only, so we can use the same distribution on all boxes. If we could have a network shared device, we'd put them all there. Sadly, our infrastructure is kind of scattered, so we have to either move .TAR files around or redo the installs to rebuild the environments.

In a few cases (e.g., PIL), we have to recompile and check the version numbers.

Answer 4

You will want easy_setup to get the eggs (roughly what Maven calls an artifact).

For setting up your environment, have a look at working-env.py

Python is not compiled but you can put all files for a project in an egg. This is done with setuptools

For CI, check this answer.