How do I debug a non-functioning PyInstaller build?

Question

I have used PyInstaller only once before, and it worked pretty straight forward with wxPython. I'm currently trying to build a different project though. The project works well when run from the command line. After it's built, however, it never launches the main window (wxPython).

I've set the debug and console flags to True in the build spec. I've also added the verbose option ([('v', '', 'OPTION')]) as specified by the PyInstaller manual. Here is the spec:

# -*- mode: python -*-
# basedir = os.path.realpath(os.path.dirname(__file__))
basedir = os.getcwd()

# Build the icons toc.
icons_toc = []
for dir in os.walk(os.path.join(basedir, 'icons')):
    for icon in dir[2]:
        icons_toc.append(
            (
                os.path.join('icons', icon),
                os.path.join(dir[0], icon),
                'DATA',
            )
        )

a = Analysis(
    ['application.py'],
    pathex=['.', './lib', '../broadpy/lib', '../broadpy/vendor'],
    hiddenimports=[],
    hookspath=None
)
a.datas += icons_toc

pyz = PYZ(a.pure)

exe = EXE(
    pyz,
    a.scripts + [('v', '', 'OPTION')],
    a.binaries,
    a.zipfiles,
    a.datas,
    name=os.path.join(
        'dist', 'Address cleaner.exe'
    ),
    debug=True,
    strip=None,
    upx=True,
    console=True
)

app = BUNDLE(
    exe,
    name=os.path.join('dist', 'Address cleaner.app')
)

Now when I run the built executable, I get this output:

C:\Users\tomas\Dropbox\Broadnet\address_cleaner>"C:\Users\tomas\Dropbox\Broadnet\address_cleaner\dist\Address cleaner.exe"
_MEIPASS2 is NULL
archivename is C:\Users\tomas\Dropbox\Broadnet\address_cleaner\dist\Address cleaner.exe
Extracting binaries
Executing self as child with Setting up to run child
Creating child process
Waiting for child process to finish...
_MEIPASS2 is C:/Users/tomas/AppData/Local/Temp/_MEI30762/
archivename is C:\Users\tomas\Dropbox\Broadnet\address_cleaner\dist\Address cleaner.exe
Already in the child - running!
manifestpath: C:/Users/tomas/AppData/Local/Temp/_MEI30762/Address cleaner.exe.manifest
Activation context created
Activation context activated
C:/Users/tomas/AppData/Local/Temp/_MEI30762/python27.dll
Manipulating evironment
PYTHONPATH=C:/Users/tomas/AppData/Local/Temp/_MEI30762;C:/Users/tomas/Dropbox/Broadnet/address_cleaner/dist
PYTHONHOME=C:/Users/tomas/AppData/Local/Temp/_MEI30762/
v
# installing zipimport hook
import zipimport # builtin
# installed zipimport hook
importing modules from CArchive
import marshal # builtin
extracted iu
import imp # builtin
import nt # builtin
extracted struct
import _struct # builtin
extracted archive
Installing import hooks
out00-PYZ.pyz
Running scripts
import zlib # builtin
import errno # builtin
import _weakref # builtin
import _codecs # builtin
import _sre # builtin
import _collections # builtin
import operator # builtin
import itertools # builtin
import _bisect # builtin
import _heapq # builtin
import thread # builtin
import math # builtin
import binascii # builtin
import _hashlib # dynamically loaded from C:\Users\tomas\AppData\Local\Temp\_MEI30762\_hashlib.pyd
import _random # builtin
import cStringIO # builtin
Traceback (most recent call last):
  File "", line 65, in 
  File "C:\Users\tomas\Downloads\pyinstaller-2.0\PyInstaller\loader\iu.py", line 386, in importHook
  File "C:\Users\tomas\Downloads\pyinstaller-2.0\PyInstaller\loader\iu.py", line 480, in doimport
  File "C:\Users\tomas\Dropbox\Broadnet\address_cleaner\build\pyi.win32\buildspec\out00-PYZ.pyz\win32com", line 5, in 
  File "C:\Users\tomas\Downloads\pyinstaller-2.0\PyInstaller\loader\iu.py", line 386, in importHook
  File "C:\Users\tomas\Downloads\pyinstaller-2.0\PyInstaller\loader\iu.py", line 459, in doimport
  File "C:\Users\tomas\Downloads\pyinstaller-2.0\PyInstaller\loader\iu.py", line 248, in getmod
  File "C:\Users\tomas\Downloads\pyinstaller-2.0\PyInstaller\loader\iu.py", line 105, in getmod
ImportError: DLL load failed: The specified module could not be found.
RC: -1 from pyi_rth_win32comgenpy
OK.
Deactivating activation context
Releasing activation context
Done
# clear __builtin__._
# clear sys.path
# clear sys.argv
# clear sys.ps1
# clear sys.ps2
# clear sys.exitfunc
# clear sys.exc_type
# clear sys.exc_value
# clear sys.exc_traceback
# clear sys.last_type
# clear sys.last_value
# clear sys.last_traceback
# clear sys.path_hooks
# clear sys.path_importer_cache
# clear sys.meta_path
# clear sys.flags
# clear sys.float_info
# restore sys.stdin
# restore sys.stdout
# restore sys.stderr
# cleanup __main__
# cleanup[1] cStringIO
# cleanup[1] __future__
# cleanup[1] _collections
# cleanup[1] encodings
# cleanup[1] site
# cleanup[1] atexit
# cleanup[1] shutil
# cleanup[1] _heapq
# cleanup[1] _weakref
# cleanup[1] abc
# cleanup[1] _bisect
# cleanup[1] _weakrefset
# cleanup[1] tempfile
# cleanup[1] binascii
# cleanup[1] sre_constants
# cleanup[1] collections
# cleanup[1] _codecs
# cleanup[1] _warnings
# cleanup[1] math
# cleanup[1] operator
# cleanup[1] fnmatch
# cleanup[1] codecs
# cleanup[1] re
# cleanup[1] _struct
# cleanup[1] thread
# cleanup[1] keyword
# cleanup[1] signal
# cleanup[1] random
# cleanup[1] itertools
# cleanup[1] encodings.aliases
# cleanup[1] exceptions
# cleanup[1] heapq
# cleanup[1] sre_compile
# cleanup[1] _sre
# cleanup[1] _random
# cleanup[1] hashlib
# cleanup[1] bisect
# cleanup[1] sre_parse
# cleanup[1] _hashlib
# cleanup[2] copy_reg
# cleanup[2] iu
# cleanup[2] os.path
# cleanup[2] archive
# cleanup[2] struct
# cleanup[2] errno
# cleanup[2] imp
# cleanup[2] _abcoll
# cleanup[2] ntpath
# cleanup[2] nt
# cleanup[2] genericpath
# cleanup[2] stat
# cleanup[2] zipimport
# cleanup[2] warnings
# cleanup[2] UserDict
# cleanup[2] types
# cleanup[2] zlib
# cleanup[2] linecache
# cleanup[2] os
# cleanup[2] marshal
# cleanup sys
# cleanup __builtin__
# cleanup ints: 41 unfreed ints
# cleanup floats: 31 unfreed floats
Back to parent...
Freeing status for C:\Users\tomas\Dropbox\Broadnet\address_cleaner\dist\Address cleaner.exe

So now I see there's a raised exception from iu.py, but I still have no idea why. I also find it quite mysterious that a python file in my Downloads/py-installer folder is run, even if I delete that folder.

So to summarize - what steps do I have to take to figure out exactly why the application crashes on startup?

---

I'm using PyInstaller 2.0 with Python 2.7.3 on Windows 8. None of those facts alone causes this error, as I'm building another project successfully.

Answer 1

Pyinstaller sometimes needs explicit references in the .spec file to correctly package dependencies.

For more information, see ensuring proper import statements so that pyinstaller recognizes them.

For instance, it is very easy to miss crucial dependencies if they are imported from outside one of your Python modules (e.g. from a jar or c++ file which pyinstaller will not read).

dependency walker could be your first line of defense to systematically tracking missing DLLs. Simply download it and then load your exe or affiliated dlls to see which ones are missing dependencies. Then, it is just a wild goose chase tracking them down and manually adding them to your directory along with the .exe (assuming your packaging in one directory).

As a side note, for pyinstaller 2.1 (python 2.7.6), I modified the pyi_importers.py file to at least try and print which module was the trouble maker when importing:

# line 409 of Pyinstaller.loader.pyi_importers.py
try: module = imp.load_module(fullname, fp, filename, self._c_ext_tuple)
except Exception as e:
    print fullname # at least tells you what module couldn't be imported
    raise e

Then, knowing where the issue occurred, you can then pin point the problem with dependency walker to root out the missing DLLs.

Answer 2

I know this answer doesn't answer how to debug issues like this, so I won't mark it as correct, but I managed to build the application and I thought I should share how. I did three things, each of which could be the cause of the error, but we'll never know:

• I replaced ActivePython 2.7.3 with the official Python 2.7.3.
• I replaced MySQLdb with the pure python drop-in replacement pymysql.
• I replaced Levenshtein with the very similar method difflib.SequenceMatcher.ratio.