Pyqt4-pyqtgraph application recursively opens new instances of itself

Question

I forked BMDanalyse for my project and have modified it.

Setup Instructions in case you need a hand. Actual question follows this.

A 64-bit system is required due to memory requirements. Please ensure you have 64bit Python 2.7.

The following setup instructions work with Pycharm on Windows or Ubuntu, though you'll want to have it running on Windows since my problem is about making an .exe:

The following packages are required. Don't forget to pick the 64bit ones. In the cases where links are provided you will not simply be able to install the packages via Pycharm.

• numpy
• pyqtgraph
• matplotlib
• PyQt4
• pillow
• scipy
• parmap

After opening the BMDanalyse project you should then be able to run SPCanalyse.py to get the app going. It has bugs and is a very early prototype. Download the following example data to make sure it is working as intended.

The following should work if you do it in order. You can of course check your IDE's console for what the app is doing (haven't added progress bars yet)

1. click file, load images, load both images. Click 'Alignment'. After the app does its thing a file called aligned.raw should appear in your Downloads folder
2. load the alignment.raw image. 'alignment.raw' will appear on the list on the right. Click on alignment.raw. Click ROIs, Add ROIs -> polygon. Draw two polygons as in the following picture: After you have your ROI's, click Analyse -> ROI Analyis. ROI.raw should appear in your downloads folder.
3. Load ROI.raw. Click on it. Click temporal filter. After a while dfoverf0_avg_framesIncl.raw should appear in your downloads folder
4. Load dfoverf0_avg_framesIncl.raw. Click on it. Screen should go black. Remove the ROI's if they are still there. Now click in random places on the black scene. What you are seeing are seed pixel correlation maps

If all those 4 steps work the app is working as intended on your system.

---

Actual Question

However, now that it is working I need a standalone .exe of this app that works without any dependencies just to show my supervisor it can be done.

If I use python SPCanalyse.py on my main script in the project it runs and my app pops up.

Update: My laptop was stolen. However in setting everything up on a clean system I'm struck with different problem. The only major thing I can think of that I did differently is I did not rely on Anaconda from Continuum analytics for package management.

If I use pyinstaller SPCanalyse.py on my main script I get an .exe and if I run that .exe firstly I notice that my icons are missing. Secondly, this happens:

When I press the alignment button (i.e. go through step 1 in the setup) the app freezes (i.e . not responding) and then repeatedly opens new isntances of itself. I'm assuming this is memory related since this operation does require a significant amount of it. If I look at how much memory the app is taking (looking only at the original "not responding" instance) I see it plateaus at exactly 1,180.8MB each time. Taking all other running processes into account though I'm still only using 28% of my 32GB memory. I have tried setting the priority of the app to high to no effect. I have also tried opening via the command prompt to see if I get any helpful error message. Sadly no.

Do note though that steps 2-3 work as intended! Step 4 results in the same recursive open-itself loop upon clicking the black scene.

I think I found the explanation of what happened here: https://github.com/rdicosmo/parmap

Of course, if you happen to have open channels, or files, or other connections that should only be used by the parent process, your program may behave in a very wierd way: as an example, do not open a graphic window before calling a Parmap primitive, and do not use this library if your program is multi-threaded!

I'm thus pretty sure I'm using parmap very very incorrectly. I remain confused why my app works if I run i via my IDE though!

Old unsolved problem that is now my thief's problem

If I use pyinstaller SPCanalyse.py on my main script the following outputs when I try to run the resultant .exe

D:\Home\Downloads\BMDanalyse\BMDanalyse>D:\Home\Downloads\BMDanalyse\BMDanalyse\
dist\SPCanalyse\SPCanalyse.exe
Traceback (most recent call last):
  File "<string>", line 21, in <module>
  File "c:\users\cornelis\appdata\local\temp\pip-build-6xhsyv\pyinstaller\PyInst
aller\loader\pyimod03_importers.py", line 389, in load_module
  File "BMDanalyse\SPCExplorer\filter_jeff.py", line 9, in <module>
  File "c:\users\cornelis\appdata\local\temp\pip-build-6xhsyv\pyinstaller\PyInst
aller\loader\pyimod03_importers.py", line 389, in load_module
  File "site-packages\image_registration\__init__.py", line 1, in <module>
  File "c:\users\cornelis\appdata\local\temp\pip-build-6xhsyv\pyinstaller\PyInst
aller\loader\pyimod03_importers.py", line 389, in load_module
  File "site-packages\image_registration\cross_correlation_shifts.py", line 4, i
n <module>
  File "c:\users\cornelis\appdata\local\temp\pip-build-6xhsyv\pyinstaller\PyInst
aller\loader\pyimod03_importers.py", line 389, in load_module
  File "site-packages\image_registration\fft_tools\__init__.py", line 3, in <mod
ule>
  File "c:\users\cornelis\appdata\local\temp\pip-build-6xhsyv\pyinstaller\PyInst
aller\loader\pyimod03_importers.py", line 389, in load_module
  File "site-packages\image_registration\fft_tools\correlate2d.py", line 2, in <
module>
  File "c:\users\cornelis\appdata\local\temp\pip-build-6xhsyv\pyinstaller\PyInst
aller\loader\pyimod03_importers.py", line 389, in load_module
  File "site-packages\image_registration\fft_tools\convolve_nd.py", line 329, in
 <module>
  File "c:\users\cornelis\appdata\local\temp\pip-build-6xhsyv\pyinstaller\PyInst
aller\loader\pyimod03_importers.py", line 389, in load_module
  File "site-packages\pytest.py", line 21, in <module>
  File "c:\users\cornelis\appdata\local\temp\pip-build-6xhsyv\pyinstaller\PyInst
aller\loader\pyimod03_importers.py", line 389, in load_module
  File "site-packages\_pytest\config.py", line 11, in <module>
  File "c:\users\cornelis\appdata\local\temp\pip-build-6xhsyv\pyinstaller\PyInst
aller\loader\pyimod03_importers.py", line 389, in load_module
  File "site-packages\_pytest\_code\__init__.py", line 2, in <module>
  File "c:\users\cornelis\appdata\local\temp\pip-build-6xhsyv\pyinstaller\PyInst
aller\loader\pyimod03_importers.py", line 389, in load_module
  File "site-packages\_pytest\_code\code.py", line 8, in <module>
  File "site-packages\py\_apipkg.py", line 125, in __makeattr
  File "site-packages\py\_apipkg.py", line 48, in importobj
ImportError: No module named _builtin
SPCanalyse returned -1

At first I thought my problem is related to No module named builtins and so I updated my setup.py to include all possible packages I could find that my project is remotely dependent on:

install_requires = [
    'pyinstaller'
    'future'
    'astropy',
    'image_registration',
    'scipy',
    'FITS_tools',
    'pywcs',
    'pyfits',
    'pytest',
    'parmap',
    'setuptools',
    'pyqtgraph',        
    'matplotlib',
    'numpy',
    'PIL',
    'SPCExplorer',
    ],

I also tried python setup.py build and I get:

D:\Home\Downloads\BMDanalyse>python setup.py build
running build
running build_py
creating build
creating build\lib
creating build\lib\BMDanalyse
copying BMDanalyse\customItems.py -> build\lib\BMDanalyse
copying BMDanalyse\matplotlib_fix.py -> build\lib\BMDanalyse
copying BMDanalyse\ordereddict.py -> build\lib\BMDanalyse
copying BMDanalyse\ROI.py -> build\lib\BMDanalyse
copying BMDanalyse\SidePanel.py -> build\lib\BMDanalyse
copying BMDanalyse\SPCanalyse.py -> build\lib\BMDanalyse
copying BMDanalyse\version.py -> build\lib\BMDanalyse
copying BMDanalyse\ViewBoxCustom.py -> build\lib\BMDanalyse
copying BMDanalyse\__init__.py -> build\lib\BMDanalyse
creating build\lib\BMDanalyse\icons
copying BMDanalyse\icons\arrow-down-2.png -> build\lib\BMDanalyse\icons
copying BMDanalyse\icons\arrow-left.png -> build\lib\BMDanalyse\icons
copying BMDanalyse\icons\arrow-right.png -> build\lib\BMDanalyse\icons
copying BMDanalyse\icons\arrow-up-2.png -> build\lib\BMDanalyse\icons
copying BMDanalyse\icons\filesave.png -> build\lib\BMDanalyse\icons
copying BMDanalyse\icons\file_add.png -> build\lib\BMDanalyse\icons
copying BMDanalyse\icons\file_copy.png -> build\lib\BMDanalyse\icons
copying BMDanalyse\icons\file_delete2.png -> build\lib\BMDanalyse\icons
copying BMDanalyse\icons\green-add3.png -> build\lib\BMDanalyse\icons
copying BMDanalyse\icons\logo.png -> build\lib\BMDanalyse\icons
copying BMDanalyse\icons\opened-folder.png -> build\lib\BMDanalyse\icons
copying BMDanalyse\icons\polygonIcon.png -> build\lib\BMDanalyse\icons
copying BMDanalyse\icons\README.txt -> build\lib\BMDanalyse\icons
copying BMDanalyse\icons\rectangularIcon.png -> build\lib\BMDanalyse\icons
copying BMDanalyse\icons\red_delete.png -> build\lib\BMDanalyse\icons
copying BMDanalyse\changeLog.txt -> build\lib\BMDanalyse
creating build\lib\BMDanalyse\sampleMedicalImages
creating build\lib\BMDanalyse\sampleMedicalImages\Implant
creating build\lib\BMDanalyse\sampleMedicalImages\Implant\XYplane
copying BMDanalyse\sampleMedicalImages\Implant\XYplane\vxray_XY_1.png -> build\l
ib\BMDanalyse\sampleMedicalImages\Implant\XYplane
copying BMDanalyse\sampleMedicalImages\Implant\XYplane\vxray_XY_11.png -> build\
lib\BMDanalyse\sampleMedicalImages\Implant\XYplane
copying BMDanalyse\sampleMedicalImages\Implant\XYplane\vxray_XY_21.png -> build\
lib\BMDanalyse\sampleMedicalImages\Implant\XYplane
copying BMDanalyse\sampleMedicalImages\Implant\XYplane\vxray_XY_31.png -> build\
lib\BMDanalyse\sampleMedicalImages\Implant\XYplane
copying BMDanalyse\sampleMedicalImages\Implant\XYplane\vxray_XY_6.png -> build\l
ib\BMDanalyse\sampleMedicalImages\Implant\XYplane
creating build\lib\BMDanalyse\sampleMedicalImages\Implant\YZplane
copying BMDanalyse\sampleMedicalImages\Implant\YZplane\vxray_YZ_1.png -> build\l
ib\BMDanalyse\sampleMedicalImages\Implant\YZplane
copying BMDanalyse\sampleMedicalImages\Implant\YZplane\vxray_YZ_11.png -> build\
lib\BMDanalyse\sampleMedicalImages\Implant\YZplane
copying BMDanalyse\sampleMedicalImages\Implant\YZplane\vxray_YZ_21.png -> build\
lib\BMDanalyse\sampleMedicalImages\Implant\YZplane
copying BMDanalyse\sampleMedicalImages\Implant\YZplane\vxray_YZ_31.png -> build\
lib\BMDanalyse\sampleMedicalImages\Implant\YZplane
copying BMDanalyse\sampleMedicalImages\Implant\YZplane\vxray_YZ_6.png -> build\l
ib\BMDanalyse\sampleMedicalImages\Implant\YZplane
creating build\lib\BMDanalyse\sampleMedicalImages\No implant
creating build\lib\BMDanalyse\sampleMedicalImages\No implant\XYplane
copying BMDanalyse\sampleMedicalImages\No implant\XYplane\noimplant_XY_1.png ->
build\lib\BMDanalyse\sampleMedicalImages\No implant\XYplane
copying BMDanalyse\sampleMedicalImages\No implant\XYplane\noimplant_XY_11.png ->
 build\lib\BMDanalyse\sampleMedicalImages\No implant\XYplane
copying BMDanalyse\sampleMedicalImages\No implant\XYplane\noimplant_XY_21.png ->
 build\lib\BMDanalyse\sampleMedicalImages\No implant\XYplane
copying BMDanalyse\sampleMedicalImages\No implant\XYplane\noimplant_XY_31.png ->
 build\lib\BMDanalyse\sampleMedicalImages\No implant\XYplane
copying BMDanalyse\sampleMedicalImages\No implant\XYplane\noimplant_XY_6.png ->
build\lib\BMDanalyse\sampleMedicalImages\No implant\XYplane
creating build\lib\BMDanalyse\sampleMedicalImages\No implant\YZplane
copying BMDanalyse\sampleMedicalImages\No implant\YZplane\noimplant_YZ_1.png ->
build\lib\BMDanalyse\sampleMedicalImages\No implant\YZplane
copying BMDanalyse\sampleMedicalImages\No implant\YZplane\noimplant_YZ_11.png ->
 build\lib\BMDanalyse\sampleMedicalImages\No implant\YZplane
copying BMDanalyse\sampleMedicalImages\No implant\YZplane\noimplant_YZ_21.png ->
 build\lib\BMDanalyse\sampleMedicalImages\No implant\YZplane
copying BMDanalyse\sampleMedicalImages\No implant\YZplane\noimplant_YZ_31.png ->
 build\lib\BMDanalyse\sampleMedicalImages\No implant\YZplane
copying BMDanalyse\sampleMedicalImages\No implant\YZplane\noimplant_YZ_6.png ->
build\lib\BMDanalyse\sampleMedicalImages\No implant\YZplane

When I go to where all this is "built" I see that a folder with scripts absolutely essential to my program is missing. Perhaps this hints at what is causing my error? See image of what running build accomplishes (i.e no SPCExplorer folder):

I don't know if this or setup.py necessarily has anything to do with why running the .exe gives me the error though

Answer 1

So the problem did indeed turn out to be due to multiprocessing. However, if you google 'multiprocessing pyinstaller' you can find this

So as it turns out

Just add multiprocessing.freeze_support() after if __name__=='__main__': in your main file (SPCanalyse.py in my case) and add import multiprocessing at the beginning of the file.

My program now runs as a standalone .exe and meets all four listed requirements