cx_Freeze - Preventing including unneeded packages

Question

I have coded a tiny python program using PyQt4. Now, I want to use cx_Freeze to create a standalone application. Everything works fine - cx_Freeze includes automatically all necessary modules; the resulting exe works.

The only problem is that cx_Freeze packs plenty of unneeded modules into the standalone. Even though I only use QtCore and QtGui, also modules like sqlite3, QtNetwork, or QtScript are included. Surprisingly, I find also PyQt5 dlls in the resulting folder. It seems to me as if cx_Freeze uses all PyQt packages that I have installed. The result is a 200Mb program - albeit I only wrote a tiny script.

How can I prevent this behaviour?

I use the following setup.py:

import sys
from cx_Freeze import setup, Executable

setup(
    name="MyProgram",
    version="0.1",
    description="MyDescription",
    executables=[Executable("MyProgram.py", base = "Win32GUI")],
)

I tried explicitely excluding some packages (although it is quite messy to exclude all unused Qt modules) adding this code:

build_exe_options = {"excludes": ["tkinter", "PyQt4.sqlite3",
                              "PyQt4.QtOpenGL4", "PyQt4.QtSql"]}

but the upper modules were still used. I also tried

build_exe_options = {"excludes": ["tkinter", "PyQt4.sqlite3",
                              "QtOpenGL4", "QtSql"]}

with the same result.

In addition to the nedless Qt packages I find also unneded folders with names like "imageformats", "tcl", and "tk". How can I include only needed files in order to keep the standalone folder and installer as small as possible?

I googled this problem for hours but only found this thread which did not help me.

I am running python 3.4.2 amd64 on windows 8.

I am happy about every solution that gives me the desired result "standalone" with a reasonable size. I tried also pyqtdeploy but ran into the error: Unknown module(s) in QT (but this is a different question).

Edit:

I am using two modules. One is the GUI class created by uic, "MyProgramGUIPreset". In this file there are the following import commands:

from PyQt4 import QtCore, QtGui
from matplotlibwidget import MatplotlibWidget

In the main module I do the following imports:

import MyProgramGUIPreset
import numpy as np
from PyQt4.QtGui import QApplication, QMainWindow, QMessageBox
import sys
from math import *

Maybe this helps to figuring out where the issue is.

Answer 1

The reason for the not working "excludes" command was that I forgot to include the build options into the setup. After adding the respective line into the code excluding works:

from cx_Freeze import setup, Executable
import sys

# exclude unneeded packages. More could be added. Has to be changed for
# other programs.
build_exe_options = {"excludes": ["tkinter", "PyQt4.QtSql", "sqlite3", 
                                  "scipy.lib.lapack.flapack",
                                  "PyQt4.QtNetwork",
                                  "PyQt4.QtScript",
                                  "numpy.core._dotblas", 
                                  "PyQt5"],
                     "optimize": 2}

# Information about the program and build command. Has to be adjusted for
# other programs
setup(
    name="MyProgram",                           # Name of the program
    version="0.1",                              # Version number
    description="MyDescription",                # Description
    options = {"build_exe": build_exe_options}, # <-- the missing line
    executables=[Executable("MyProgram.py",     # Executable python file
                            base = ("Win32GUI" if sys.platform == "win32" 
                            else None))],
)

This decreased the program size from 230MB to 120MB. Nevertheless, I did not find a nice way of excluding all unneeded packages. By trial and error (deleting the biggest files in the build folder test-wise) I figured out which classes I can exclude.

I tried whether the matplotlib backends cause the problem and finally figured out that this is not the case. Nontheless, if anybody needs code to exclude all modules of a certain name scheme in a particular folder except some special ones, he may adjust the following to his needs:

mplBackendsPath = os.path.join(os.path.split(sys.executable)[0],
                        "Lib/site-packages/matplotlib/backends/backend_*")

fileList = glob.glob(mplBackendsPath)

moduleList = []

for mod in fileList:
    modules = os.path.splitext(os.path.basename(mod))[0]
    if not module == "backend_qt4agg":
        moduleList.append("matplotlib.backends." + modules)

build_exe_options = {"excludes": ["tkinter"] + moduleList, "optimize": 2}

I would be happy about more elegant solutions. Further ideas are still welcome. Nevertheless, I regard the problem as solved for me.

Answer 2

I was having a similar problem on a very simple PyQt4 Gui for a small database where the program was 58Mb for a small amount of code, the problem being that the entire PyQt4 folder was being included in the program.

The article here refers to using zip_include_packages in your options to exclude files or to compress them to reduce the file size.

I excluded the entire PyQt4 folder and then included the bits I needed as shown below and it reduced the whole package to 16Mb automatically

options = {
'build_exe': {
    'packages':packages,
    'zip_include_packages':'PyQt4',
    'includes':['PyQt4.QtCore','PyQt4.QtGui','sqlite3','sys','os'],
},

Not sure it is the right way to do it but seems to have no negative impact on my program as of yet

Answer 3

This is how I optimized my executable to the minimum file size

from cx_Freeze import setup, Executable
import subprocess
import sys


NAME = 'EXE NAME'
VERSION = '1.0'
PACKAGES = ['pygame', ('import_name', 'package_name')]
# if names are same just have a string not a tuple
installed_packages = subprocess.check_output([sys.executable, '-m', 'pip', 'freeze']).decode('utf-8')
installed_packages = installed_packages.split('\r\n')
EXCLUDES = {pkg.split('==')[0] for pkg in installed_packages if pkg != ''}
EXCLUDES.add('tkinter')
for pkg in PACKAGES:
    if type(pkg) == str: EXCLUDES.remove(pkg)
    else: EXCLUDES.remove(pkg[1])


executables = [Executable('main.py', base='Win32GUI', icon='Resources/Jungle Climb Icon.ico', targetName=NAME)]

setup(
    name=NAME,
    version=VERSION,
    description=f'{NAME} Copyright 2019 AUTHOR',
    options={'build_exe': {'packages': [pkg for pkg in PACKAGES if type(pkg) == str else pkg[0]],
                           'include_files': ['FOLDER'],
                           'excludes': EXCLUDES,
                           'optimize': 2}},
    executables=executables)

Answer 4

You'll need to create a list that scan all package in python , then remove your package in use in the list, finally assign excludes option with that list: (setup.py)

APP_NAME    = "Hezzo";      ## < Your App's name
Python_File = "app.py";     ## < Main Python file to run
Icon_Path   = "./icon.ico"; ## < Icon
Import      = ["tkinter"];  ## < Your Imported modules (cv2,numpy,PIL,...)

################################### CX_FREEZE IGNITER ###################################
import sys, pkgutil;
from cx_Freeze import setup, Executable;

BasicPackages=["collections","encodings","importlib"] + Import;
def AllPackage(): return [i.name for i in list(pkgutil.iter_modules()) if i.ispkg]; # Return name of all package
def notFound(A,v): # Check if v outside A
    try: A.index(v); return False;
    except: return True;
build_exe_options = {
    "includes": BasicPackages,
    "excludes": [i for i in AllPackage() if notFound(BasicPackages,i)],
}
setup(  name = APP_NAME,
        options = {"build_exe": build_exe_options},
        executables = [Executable(Python_File, base='Win32GUI', icon=Icon_Path, targetName=APP_NAME)]
);

Example app.py:

import tkinter as tk;
windowTk = tk.Tk();
windowTk.mainloop();

In some cases like creating light weight portable app, you can remove extra tcl module file and gear in the lib folder after export. It help reduce some more mega bytes (my app from 200MB decrease to 16MB then 10MB).

And you can use WinRAR to create portable program (SFX archive with option). It's usually reduce 45% more.