cxfreeze missing distutils module inside virtualenv

Question

When running a cxfreeze binary from a python3.2 project I am getting the following runtime error:

/project/dist/project/distutils/__init__.py:13: UserWarning: The virtualenv distutils package at %s appears to be in the same location as the system distutils?
Traceback (most recent call last):
  File "/home/chrish/.virtualenvs/project/lib/python3.2/distutils/__init__.py", line 19, in <module>
    import dist
ImportError: No module named dist

Correspondingly there are several distutils entries in the missing modules section of the cxfreeze output:

? dist imported from distutils
? distutils.ccompiler imported from numpy.distutils.ccompiler
? distutils.cmd imported from setuptools.dist
? distutils.command.build_ext imported from distutils
? distutils.core imported from numpy.distutils.core
...

I've tried forcing distutils to be included as a module, by both importing it in my main python file and by adding it to a cxfreeze setup.py as:

options = {"build_exe": {"packages" : ["distutils"]} },

Neither approach worked. It seems likely that I've somehow broken the virtualenv [as distutils seems fundamental and the warning regarding the location of distutils], repeating with a clean virtualenv replicated the problem.

It may be worth noting that I installed cx-freeze by running $VIRTUAL_ENV/build/cx-freeze/setup.py install as it doesn't install cleanly in pip.

Answer 1

Found another workaround which enables you to still use a virtualenv when freezing.

The workaround is to exclude distutils and add the package from the original interpreter (not from the virtualenv) manually.

# contents of setup.py
from cx_Freeze import setup, Executable

import distutils
import opcode
import os

# opcode is not a virtualenv module, so we can use it to find the stdlib; this is the same
# trick used by distutils itself it installs itself into the virtualenv
distutils_path = os.path.join(os.path.dirname(opcode.__file__), 'distutils')
build_exe_options = {'include_files': [(distutils_path, 'distutils')], "excludes": ["distutils"]}

setup(
    name="foo",
    version="0.1",
    description="My app",
    options={"build_exe": build_exe_options},
    executables=[Executable("foo_main.py", base=None)],
)

Credit to Bruno Oliveira for the answer on github
Full answer in gist: https://gist.github.com/nicoddemus/ca0acd93a20acbc42d1d

Answer 2

Summarising my comments:

The copy of distutils in the virtualenv is doing some bizarre things which confuse cx_Freeze. The simple workaround is to freeze outside a virtualenv, so that it uses the system copy of distutils.

On Ubuntu, Python 2 and 3 co-exist happily: just use python3 to do anything with Python 3. E.g. to install cx_Freeze under Python 3: python3 setup.py install.

Answer 3

I have found a workaround about the distutils problem when freezing from within a virtualenv that might help others.

First make sure to exclude distutils from your build:

build_exe_options = {'excludes': ['distutils']}

Second declare this function in your setup.py file:

def copy_distutils_to_build_dir(build_dir):
    # the code below was obtained from the distutils.py file created by
    # virtualenv
    import opcode
    dirname = os.path.dirname
    distutils_path = os.path.join(os.path.dirname(opcode.__file__), 'distutils')
    target_dir = os.path.join(build_dir, 'distutils')
    if os.path.isdir(target_dir):
        shutil.rmtree(target_dir)
    shutil.copytree(distutils_path, target_dir)

Finally, call the function after calling setup() in your setup.py:

setup(...)
copy_distutils_to_build_dir(join('build', 'exe.win32-3.4'))

This will copy the entire distutils package from the original interpreter to the directory containing your frozen executable.

Hackish and ugly, but it works. I would love to hear ideas for improvement though.