How can I make jenkins run "pip install"?

Question

I have a git repo and would like to get jenkins to clone it then run

virtualenv venv --distribute
/bin/bash venv/source/activate
pip install -r requirements.txt
python tests.py

The console output from jenkins is:

+ virtualenv venv --distribute
New python executable in venv/bin/python
Installing distribute..........................done.
Installing pip...............done.
+ /bin/bash venv/bin/activate
+ pip install -r requirements.txt
Downloading/unpacking flask (from -r requirements.txt (line 1))
  Running setup.py egg_info for package flask

SNIP

creating /usr/local/lib/python2.7/dist-packages/flask

error: could not create '/usr/local/lib/python2.7/dist-packages/flask': Permission denied

----------------------------------------
Command /usr/bin/python -c "import setuptools;__file__='/var/lib/jenkins/workspace/infatics-website/build/flask/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --single-version-externally-managed --record /tmp/pip-hkdBAi-record/install-record.txt failed with error code 1
Storing complete log in /home/jenkins/.pip/pip.log
Build step 'Execute shell' marked build as failure
Finished: FAILURE

I've tried adding sudo before the command, but it's doesn't work either:

+ sudo pip install -r requirements.txt
sudo: no tty present and no askpass program specified
Sorry, try again.
sudo: no tty present and no askpass program specified
Sorry, try again.
sudo: no tty present and no askpass program specified
Sorry, try again.
sudo: 3 incorrect password attempts
Build step 'Execute shell' marked build as failure
Finished: FAILURE

Any ideas how to get around this? Also when I run pip install -r requirement.txt in a terminal as the jenkins user it doesn't need sudo permission. Can I get jenkins (the process) to run as the jenkins user?

Answer 1

The fact that you have to run use sudo to run pip is a big warning that your virtual environment isn't working. The build output shows that pip is installing the requirements in your system site-packages directory, which is not the way virtualenv works.

Your build script doesn't actually preserve the activated virtual environment. The environment variables set by the activate script are set in a child bash process and are not propagated up to the build script. You should source the activate script instead of running a separate shell:

virtualenv venv --distribute
. venv/bin/activate 
pip install -r requirements.txt
python tests.py

If you're running this as one build step, that should work (and install your packages in venv). If you want to add more steps, you'll need to set the PATH environment variable in your other steps. You're probably better off providing full paths to pip and python to ensure you're not dependent on system package installations.