How do I maintain Python packages on OS X Lion?

Question

What is the correct way to install and update Python packages on OS X Mountain Lion, using Apple's built in Python?

I've tried all everything I can find here and in the documentation for various packages but keep encountering all kinds of errors that result in failed or aborted installations or updates. Notably, with pip I frequently get errors like

RuntimeError: maximum recursion depth exceeded in cmp

or

SError: [Errno 24] Too many open files: '/Library/Python/2.7/site-packages'

and some commands pip commands (like list) just don't seem to exist at all.

I have an uptodate version of pip that works fine with some packages, and have the latest Xcode command line tools installed.

The only thing that seems to work — and it works all the time — is

python setup.py install

Is this the correct way to maintain and install python packages on Lion? Is there any reason not to use this method as my primary method for keeping packages uptodate? Should pip be working for me, or does it not work with Apple's Python?

---

UPDATE: I spoke too soon. I also have problems with

python setup.py install

Occasionally (often) I'll get:

error: /Library/Python/2.7/site-packages: Too many open files

When nothing is running in a fresh Terminal, I get about 50 when I

lsof | grep python | wc -l

and if I quit Dropbox, which seems to be the source of these, and reduce the number to 0, I still get "Too many open files" for most package installation operations, with whatever tool I use, even after a reboot of my system.

Why am I getting a "too many open files" error so often?

Answer 1

You should be able to maintain exactly as you're doing, but likely need to update your PYTHONPATH to somewhere outside of /Library, hence the reason everyone else is mentioning that and/or using homebrew to install python.

I install python packages with pip and maintain them using:

https://github.com/nvie/pip-tools