python3 pip broken since system update

Question

After updating one of my Ubuntu 16.04 machines on AWS, python/pip appears to be broken.

ubuntu@host:~$ pip install celery
Traceback (most recent call last):
  File "/home/ubuntu/virtualenvs/myenv/bin/pip", line 7, in <module>
    from pip import main
  File "/home/ubuntu/virtualenvs/myenv/lib/python3.6/site-packages/pip/__init__.py", line 43, in <module>
    from pip.utils import get_installed_distributions, get_prog
  File "/home/ubuntu/virtualenvs/myenv/lib/python3.6/site-packages/pip/utils/__init__.py", line 23, in <module>
    from pip.locations import (
  File "/home/ubuntu/virtualenvs/myenv/lib/python3.6/site-packages/pip/locations.py", line 9, in <module>
    from distutils import sysconfig
  File "/home/ubuntu/virtualenvs/myenv/lib/python3.6/distutils/__init__.py", line 25, in <module>
    from distutils import dist, sysconfig
ImportError: cannot import name 'dist'

ubuntu@host:~$ python3.6 -m pip install celery
Traceback (most recent call last):
  File "/usr/lib/python3.6/runpy.py", line 183, in _run_module_as_main
    mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
  File "/usr/lib/python3.6/runpy.py", line 142, in _get_module_details
    return _get_module_details(pkg_main_name, error)
  File "/usr/lib/python3.6/runpy.py", line 109, in _get_module_details
    __import__(pkg_name)
  File "/home/ubuntu/virtualenvs/myenv/lib/python3.6/site-packages/pip/__init__.py", line 43, in <module>
    from pip.utils import get_installed_distributions, get_prog
  File "/home/ubuntu/virtualenvs/myenv/lib/python3.6/site-packages/pip/utils/__init__.py", line 23, in <module>
    from pip.locations import (
  File "/home/ubuntu/virtualenvs/myenv/lib/python3.6/site-packages/pip/locations.py", line 9, in <module>
    from distutils import sysconfig
  File "/home/ubuntu/virtualenvs/myenv/lib/python3.6/distutils/__init__.py", line 25, in <module>
    from distutils import dist, sysconfig
ImportError: cannot import name 'dist'

The system runs python3.6 installed through the jonathonf/python-3.6 ppa. Before the system update (apt-get update && apt-get upgrade) everything worked as expected. I also verified this on other machines I have not updated yet.

I have also had no luck installing python3.6 in a virtual environment on a fresh machine, so I guess a recent release or update broke something here?

---

edit:

I am also running into similar problems when installing on a fresh ubuntu:

$ sudo apt-get update
$ sudo apt-get upgrade
$ sudo add-apt-repository jonathonf/python-3.6
$ sudo apt-get update
$ sudo apt-get install python3.6

$ stat /usr/lib/python3.6/distutils/dist.py
stat: cannot stat '/usr/lib/python3.6/distutils/dist.py': No such file or directory

$ ll /usr/lib/python3.6/distutils/
total 36
drwxr-xr-x  2 root root  4096 May  3 15:55 ./
drwxr-xr-x 28 root root 12288 May  3 15:44 ../
-rw-r--r--  1 root root   236 May  3 11:50 __init__.py
-rw-r--r--  1 root root 12345 May  3 11:50 version.py

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 16.04.4 LTS
Release:        16.04
Codename:       xenial

Answer 1

I've tested the PPA in a fresh docker container and it looks like the package is broken, all distutils modules are missing after installation. Looks like the package was rebuilt recently; maybe this is a new issue.

You can open a bug and wait until the PPA maintainer fixes it, but aside from that, there's not much you can do. You can of course try fiddling the installation:

$ wget https://www.python.org/ftp/python/3.6.5/Python-3.6.5.tgz
$ tar xvf Python-3.6.5.tgz
$ rsync -av Python-3.6.5/Lib/distutils/ /usr/lib/python3.6/distutils/

Of course, this is a very dirty stuff because I don't know what files should the correct package from that PPA have. Thus, I can give you no guarantee whatsoever whether this will work for you and not introduce any new bugs, but at least I could get pip running again and could install a package.

Answer 2

This solves the problem for me on Ubuntu and Linux Mint. First confirm that python3.6 is installed through ppa:jonathonf/python-3.6 by running the commands:

ls /etc/apt/sources.list.d
sudo rm -i /etc/apt/sources.list.d/jonathonf-python-3_6-xenial.list

Then add the deadsnakes PPA using:

sudo add-apt-repository ppa:deadsnakes/ppa
sudo apt-get update

And finally either remove/install or reinstall python3.6

sudo apt-get remove python3.6
sudo apt-get install python3.6

I also ran into unmet dependencies when reinstalling python. This was solved by manually removing libpython3.6-minimal. Hope this helps!