PIP install fails in python3 venv with permission denied in /tmp folder

Question

There seems to be several ways to get to the same error, I'm describing most typical.

I've compiled python 3.6.2 on shared hosting server where I have no sudo rights (nevertheless it works nicely in cgi-bin). Later I've found it also happens on Ubuntu 16.04. I'm trying:

my-python3-path/bin/python3 -m venv my-venv-folder

This fails at the end with

Error: Command '['...my-venv-folder/bin/python3', '-Im', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit status 1.

I can try another way:

my-python3-path/bin/python3 -m venv --without-pip my-venv-folder

which succeeds, and then

my-venv-folder/bin/python -m ensurepip

(or with more options as suggested by result above) - it anyway ends with:

OSError: [Errno 22] Invalid argument: '/tmp/tmpt2cathtr'

and somewhat above the stacktrace ending with:

PermissionError: [Errno 13] Permission denied: '/tmp/pip-build-exot00s_'

Though in this way all necessary scripts (e.g. activate) are created in venv before pip, and pip files themselves seem to be present and working, until I really try install anything. Then the similar error exists.

Changing TMPDIR doesn't seem to help - the same /tmp folder is used.

The same or similar error occurred during make install by the way...

Subfolders in question (ones in /tmp have d-w------- access rights, so no wonder they couldn't be accessed).

If anyone encountered this problem, please share if you have find solution to get rid of it (though at present state the last variant looks like workaround), for I failed to find the solution with google. Sorry if description is not very coherent, for I'm not a python man and various attempts and googling made me a bit dizzy. Thanks in advance!

UPD: it seems to happen that temporary directories are created with rwx rights for user, but later the mode becomes -w- and rmtree fails. current workaround seems to be comment out rmtree calls in such places (several of them)...

Answer 1

I think I've found a fix/workaround which works.

The trouble is caused by strange change in /tmp/pip... subdirectories' access rights - they end up as "write-only" so rmtree being called at various stages by pip fails miserably.

My solution is:

1. Setup venv without pip
2. Download pip package from pypi manually and put it to venv
3. Fix rmtree in pip/utils
4. Use pip to install setuptools
5. Now it's working (though should be called via python)

Minor flaw of this approach is that temporary folders are not deleted automatically. I either remove them (chmodding recursively first) or wait for reboot.

---

In details it is like this:

Create virtualenv and start it:

path-to-python3/bin/python3 -m venv my-venv --without-pip
source my-venv/bin/activate

Let's change to directory where libs should be installed

cd my-venv/lib/python3.6/site-packages

download pip (copy link to .whl from the official page https://pypi.python.org/pypi/pip)

wget https://pypi.python.org/packages/.../pip-9.0.1-py2.py3-none-any.whl

now extract pip directory from it and remove all other stuff

unzip pip-9.0.1...whl
rm *.whl
rm -r pip-9.0.1.dist-info

Now open file pip/utils/__init__.py and change the header of rmtree function:

nano pip/utils/__init__.py

find the line def rmtree(dir, ignore_errors=False): and change False to True. then exit with Ctrl-X, Yes.

to work properly pip wants to have setuptools, luckily we now can install them with pip itself:

python -m pip install setuptools

and now it is ready to be used for managing everything what we need:

python -m pip install django
python -m pip freeze

P.S. Even more intelligent approach would be to try chown recursively inside utils/rmtree.