What's the best practice for installing development versions of Python modules in Anaconda?

Question

I am using the Anaconda python distribution on a Mac. I would like to play with the latest version of the matplotlib source code on Github, make a few changes, and see how it runs. But most of the time I would simply like to use the normal version of matplotlib that comes with Anaconda Python, so I would like a way to switch back and forth easily.

The matplotlib documentation has a good description of the branching and pull-request workflow, but it's not clear to me how I actually install and use the development version of matplotlib in a way that will preserve my working Python implementation.

My guess is that I want to set up an environment that contains the latest matplotlib version and it's dependencies and switch between that environment and the normal root environment. But when I use python setup.py develop to install the development version of matplotlib, it seems to install to both environments.

So, what is the best practice for working with the development version of a Python package from GitHub?

Answer 1

As you mentioned in your question, conda env is capable of maintaining separate Python environments for development versions of whichever packages you want to work on.

---

I'm not quite sure why you are finding that python setup.py develop is installing the dev version of matplotlib to your root environment. Perhaps you created a new environment, but didn't activate it before installing the dev version of matplotlib? For example:

~$ conda create --name matplotlib-dev --clone root
Fetching package metadata: ....
src_prefix: '/home/alistair/anaconda'
dst_prefix: '/home/alistair/anaconda/envs/matplotlib-dev'
Packages: 165
Files: 32
Linking packages ...
[      COMPLETE      ]|#####################################################| 100%
#
# To activate this environment, use:
# $ source activate matplotlib-dev
#
# To deactivate this environment, use:
# $ source deactivate
#
~$ conda info --envs 
# conda environments:
#
matplotlib-dev           /home/alistair/anaconda/envs/matplotlib-dev
root                  *  /home/alistair/anaconda

At this point I have created a matplotlib-dev environment but I haven't activated it yet, so installing any new packages would still modify my root environment.

~$ source activate matplotlib-dev
discarding /home/alistair/anaconda/bin from PATH
prepending /home/alistair/anaconda/envs/matplotlib-dev/bin to PATH

(matplotlib-dev)~$ conda info --envs 
# conda environments:
#
matplotlib-dev        *  /home/alistair/anaconda/envs/matplotlib-dev
root                     /home/alistair/anaconda

---

In any case, using setuptools directly (i.e. python setup.py install or python setup.py develop) is no longer recommended, and might not be supported by future versions of numpy etc..

The preferred method is to use pip install <path>, or pip install -e <path> if you want an "editable" installation (similar to what python setup.py develop gives you):

(matplotlib-dev)~$ pip install -e git+git://github.com/matplotlib/matplotlib.git#egg=matplotlib-dev
Obtaining matplotlib from git+git://github.com/matplotlib/matplotlib.git#egg=matplotlib-dev
  Cloning git://github.com/matplotlib/matplotlib.git to ./src/matplotlib
...
Installing collected packages: matplotlib
  Running setup.py develop for matplotlib
Successfully installed matplotlib-1.5.0+337.g595868a

(matplotlib-dev)~$ python -c "import matplotlib; print(matplotlib.__version__)"
1.5.0.post337+g595868a

(matplotlib-dev)~$ source deactivate
discarding /home/alistair/anaconda/envs/matplotlib-dev/bin from PATH

~$ python -c "import matplotlib; print(matplotlib.__version__)"
1.4.3

Note the #egg= part, which tells pip to install the source to ./src/matplotlib-dev. Instead of a git URI you could also pass pip the path to a local source directory if you've already got a local copy that you're currently working on.

It should also be possible to use conda develop <path> instead of pip install -e <path>, although conda doesn't seem to offer integrated VCS support like pip does.