`pip install --upgrade pip` fails inside a Windows virtualenv with "Access denied"

Question

On Windows, if you try to use pip to upgrade itself, inside a virtualenv, you may get a mysterious "access is denied" error. For instance:

D:\scratch\> C:\Program Files\Python\3.7.4\x64\python.exe -m venv D:\scratch\my-venv
D:\scratch\> D:\scratch\my-venv\Scripts\activate
(my-venv) D:\scratch\> pip install --upgrade pip

Collecting pip
  Downloading pip-19.3.1-py2.py3-none-any.whl (1.4MB)
Installing collected packages: pip
  Found existing installation: pip 19.0.3
    Uninstalling pip-19.0.3:
Could not install packages due to an EnvironmentError: 
  [WinError 5] Access is denied: 'd:\\scratch\\my-venv\\scripts\\pip.exe'
Consider using the `--user` option or check the permissions.

This happens whether or not the command prompt has administrative privileges. We know we have write access to everything inside d:\scratch\my-venv, because we just created it with the initial python -m venv command. The advice to use the --user option is unhelpful, since we want to upgrade the version of pip inside the virtualenv, which --user will not do.

What could be wrong, and what is the correct way to upgrade pip inside a virtualenv on Windows?

Answer 1

I don't know if this is the only reason this can happen, but notice that the "Access is denied" error points at d:\scratch\my-venv\scripts\pip.exe. pip is trying to replace itself, and Windows doesn't allow you to modify a running EXE file in any way.

A workaround for this specific problem is to use python -m pip install --upgrade pip instead. This way, pip.exe is not running, so Windows will allow it to be replaced. This action doesn't try to overwrite d:\scratch\my-venv\scripts\python.exe, and Windows doesn't care what pip does to all the other files belonging to the pip package.

See https://github.com/pypa/pip/issues/188 and https://github.com/pypa/pip/issues/1299 for further information.