cx-freeze, runpy and multiprocessing - multiple paths to failure

Question

This is a bit of a complex one, and may take some of your time.

The basic problem is, that on linux (Ubuntu in my test case) a cx-freeze'd version of my program (Omnitool) is not able to create subprocesses. It works on Windows 7, however. Or when running directly from source code. Unfortunately it's not as simple as forgetting freeze_support.

The Problem

Default behaviour of starting a subprocess, is that the X Server crashes. Specifically like this:

XIO:  fatal IO error 11 (Resource temporarily unavailable) on X server ":0"
      after 23 requests (23 known processed) with 0 events remaining.
[xcb]Unknown sequence number while processing queue 
[xcb] Most likely this is a multi-threaded client and XInitThreads has not been called 
[xcb] Aborting, sorry about that. 
Omnitool: ../../src/xcb_io.c:274: poll_for_event: Assertion `!xcb_xlib_threads_sequence_lost' failed. 
Aborted (core dumped)

There is no python traceback. I tried to manually call XInitThreads with ctypes, it returns 0 for successfully set, but crashes regardless. Shorty before the crash, the pygame UI bugs out, so I expect that something there faults.

Now, setting multiprocessing.set_start_method() changes problems around: "forkserver" gives me this lovely traceback, that tells me nothing:

Pastebin Example

Setting it to spawn, instead, just makes it not do anything. The process starts, and goes through __main__, as I can prove with prints, but never enters the target function for the subprocess.

Trying yourself

Make sure you have Ubuntu or comparable Linux with python 3.4. Then to get all dependencies:

Download omnitool as zip or clone from git: https://github.com/Berserker66/omnitool requirements.txt in the following code is from Omnitool.

sudo apt-get update -qq
sudo apt-get install --fix-missing mercurial subversion python3-dev python3-numpy libav-tools libsdl-image1.2-dev libsdl-mixer1.2-dev libsdl-ttf2.0-dev libsmpeg-dev libsdl1.2-dev libportmidi-dev libswscale-dev libavformat-dev libavcodec-dev python3-pip
pip3 install -r requirements.txt
pip3 install cx_Freeze

To freeze, run omnisetup.py with the buildargument.

I've also tried freezing with pyinstallers python 3 fork, which fails very similar. The spec file isn't git tracked though.

Answer 1

I used cx_freeze for a project at work. I'm not sure if this is your problem... but I was using the Anaconda distribution, and cx_freeze was not properly gathering the .dll's that I needed for my project.

The solution was to:

1. Install a plane version of Python
2. make an environment with the packages that I needed for that project
3. Run cx_freeze.

Magically, all of the problems went away and everything compiled like it was supposed too.