How to add dynamic python modules to PyInstaller's specs?

Question

I'm trying to figure out how to load dynamic/hidden imports with PyInstaller, so far I got this simple structure:

First of all, I got a framework package added into my PYTHONPATH living in d:\Sources\personal\python\framework

That package is used by many of my python projects, in particular, it's used with the below simple project I want to package

Main project

├───data        <- Pure static data
├───plugins     <- Dynamic modules which uses framework's modules                                      
├───resources   <- Static data+embedded (generated by pyqt), used by plugins
│   ├───css                                              
│   ├───images                    
|   resources.py
|   resources.qrc                       
main.py         <- Uses framework's modules to load plugins dynamically

My spec file looks like this:

# -*- mode: python -*-

block_cipher = None


a = Analysis(['main.py'],
             pathex=['d:\\sources\\personal\\python\\pyqt\\pyshaders'],
             binaries=None,
             datas=[],
             hiddenimports=[],
             hookspath=[],
             runtime_hooks=[],
             excludes=[],
             win_no_prefer_redirects=False,
             win_private_assemblies=False,
             cipher=block_cipher)

##### include mydir in distribution #######
def extra_datas(mydir):
    def rec_glob(p, files):
        import os
        import glob
        for d in glob.glob(p):
            if os.path.isfile(d):
                files.append(d)
            rec_glob("%s/*" % d, files)
    files = []
    rec_glob("%s/*" % mydir, files)
    extra_datas = []
    for f in files:
        extra_datas.append((f, f, 'DATA'))

    return extra_datas
###########################################

a.datas += extra_datas('data')

pyz = PYZ(a.pure, a.zipped_data,
             cipher=block_cipher)
exe = EXE(pyz,
          a.scripts,
          exclude_binaries=True,
          name='main',
          debug=False,
          strip=False,
          upx=True,
          console=True )
coll = COLLECT(exe,
               a.binaries,
               a.zipfiles,
               a.datas,
               strip=False,
               upx=True,
               name='main')

The problem comes when I try to add hidden imports to the Analysis(... , hiddenimports=[], ...)'s hiddenimports list, I've tried so far this:

• hiddenimports=['d:\\sources\\personal\\python\\pyqt\\plugins']
• hiddenimports=['d:\\sources\\personal\\python\\pyqt\\plugins\\*']
• hiddenimports=['plugins']

Also tried listing as individual files with absolute paths:

hiddenimports=[
    'd:\\sources\\personal\\python\\pyqt\\pyshaders\\plugins\\api.py',
    'd:\\sources\\personal\\python\\pyqt\\pyshaders\\plugins\\config.py',
    'd:\\sources\\personal\\python\\pyqt\\pyshaders\\plugins\\plugins_actions.py',
    'd:\\sources\\personal\\python\\pyqt\\pyshaders\\plugins\\plugins_dialogs.py',
    'd:\\sources\\personal\\python\\pyqt\\pyshaders\\plugins\\plugins_docks.py',
    'd:\\sources\\personal\\python\\pyqt\\pyshaders\\plugins\\plugins_post_init.py',
    'd:\\sources\\personal\\python\\pyqt\\pyshaders\\plugins\\plugins_toolbar.py'
]

And also tried to load them as module packages (__init__.py lives in plugins folder)

hiddenimports=[
    'plugins.api',
    'plugins.config',
    'plugins.plugins_actions',
    'plugins.plugins_dialogs',
    'plugins.plugins_docks',
    'plugins.plugins_post_init',
    'plugins.plugins_toolbar'
]

Also tried the collect_submodules

hiddenimports=collect_submodules('plugins')

None of these attempts worked and the folder plugin isn't being added properly to the dist (when i say 'properly' i guess pyinstaller will analize the imports used by these hidden plugins analizing recursively their dependencies and copying the *.pyc files)... So, I'd like to know how can i add properly "hidden" modules which are being loaded dynamically to the pyinstaller's spec.

Answer 1

I've had the problem alike packing PyQt application with Py2Exe (I've also struggled with PyInstaller and cx_freeze, but only py2exe helped me).

Here's the detailed solution. I've added them explicitly as:

data_files += [('source', glob('source/*.py'),)]
setup(
      data_files=data_files,
      ....  # other options
      windows=[
        {
          "script": "launcher.py",
          "icon_resources": [(0, "resources/favicon.ico")]
        }
     )

Then I import them and call. Hope this approach will be useful.