Use pyunpack inside an executable file made with pyinstaller in combination with try except

Question

I have a strange behaviour for pyunpack, a package for unpacking, inside an executable.

I want to do the following thing:

I have a .7z type of file whose ending is not in .7z but in .sent.

First I try to unzip it the direct way, which leads to an expected error that is caught.

Inside this error catching, I am first adding the .7z extension, then I am unzipping the file properly into a folder called "grog", then I give the zipped file its original name back.

Here is the code below:

# test.py
from os.path import abspath, join, exists, dirname
from os import rename, mkdir
from shutil import copy
import multiprocessing
import pyunpack

multiprocessing.freeze_support()
print(0)
name = "file_to_be_unzipped.sent"
print("a")
path = "C:\\Users\\myname\\eclipse-workspace-tms\\test_unzip_exe"
print(abspath("."))
print("b")
unzip_dest = join(path, "grog")
if not exists(unzip_dest):
    mkdir(unzip_dest)
print("c")
name = join(path, name)
print("d")
print("e")
try:
    print(1)
    pyunpack.Archive(name).extractall(unzip_dest)
    print(2)
except pyunpack.PatoolError as pterr:
    print(3)
    temp_f_name = name + ".7z"
    print(4)
    rename(name, temp_f_name)
    try:
        print(5)
        pyunpack.Archive(temp_f_name).extractall(unzip_dest)
        print(6)
        rename(temp_f_name, name)
        print(7)
    except pyunpack.PatoolError as pterr2:
        # removing useless 7z extension
        print(8)
        rename(temp_f_name, name)
        print(9)
        # Case when the file is already unzipped
        if str(pterr2).find("Is not archive"):
            print(10)
            copy(name, unzip_dest)
            print(11)
        print(12)
except ValueError as v:
    print(13)
    print(v)
    print(14)

When I launch the script test.py, I get the expected behaviour:

0
a
C:\Users\myname\eclipse-workspace-tms\test_unzip_exe
b
c
d
e
1
3
4
5
6
7

then I build the executable with the following command line:

pyinstaller --log-level=DEBUG test.spec

and the following spec file:

# -*- mode: python ; coding: utf-8 -*-

block_cipher = None

import pyunpack
import patoolib
from pyunpack import Archive, PatoolError
from patoolib.programs import ar
from patoolib.programs import  arc
from patoolib.programs import  archmage
from patoolib.programs import  arj
from patoolib.programs import  bsdcpio
from patoolib.programs import  bsdtar
from patoolib.programs import  bzip2
from patoolib.programs import  cabextract
from patoolib.programs import  chmlib
from patoolib.programs import  clzip
from patoolib.programs import  compress
from patoolib.programs import  cpio
from patoolib.programs import  dpkg
from patoolib.programs import  flac
from patoolib.programs import  genisoimage
from patoolib.programs import  gzip
from patoolib.programs import  isoinfo
from patoolib.programs import  lbzip2
from patoolib.programs import  lcab
from patoolib.programs import  lha
from patoolib.programs import  lhasa
from patoolib.programs import  lrzip
from patoolib.programs import  lzip
from patoolib.programs import  lzma
from patoolib.programs import  lzop
from patoolib.programs import  mac
from patoolib.programs import  nomarch
from patoolib.programs import  p7azip
from patoolib.programs import  p7rzip
from patoolib.programs import  p7zip
from patoolib.programs import  pbzip2
from patoolib.programs import  pdlzip
from patoolib.programs import  pigz
from patoolib.programs import  plzip
from patoolib.programs import  py_bz2
from patoolib.programs import  py_echo
from patoolib.programs import  py_gzip
from patoolib.programs import  py_lzma
from patoolib.programs import  py_tarfile
from patoolib.programs import  py_zipfile
from patoolib.programs import  rar
from patoolib.programs import  rpm
from patoolib.programs import  rpm2cpio
from patoolib.programs import  rzip
from patoolib.programs import  shar
from patoolib.programs import  shorten
from patoolib.programs import  star
from patoolib.programs import  tar
from patoolib.programs import  unace
from patoolib.programs import  unadf
from patoolib.programs import  unalz
from patoolib.programs import  uncompress
from patoolib.programs import  unrar
from patoolib.programs import  unshar
from patoolib.programs import  unzip
from patoolib.programs import  xdms
from patoolib.programs import  xz
from patoolib.programs import  zip
from patoolib.programs import  zoo
from patoolib.programs import  zopfli
from patoolib.programs import  zpaq


# from pyunpack import Archive, PatoolError

a = Analysis(['test.py'],
             pathex=['C:\\Users\\myname\\eclipse-workspace-tms\\test_unzip_exe'],
             binaries=[],
             datas=[],
             hiddenimports=['pyunpack', 'patoolib',
                             'patoolib.programs.ar',
                             'patoolib.programs.arc',
                             'patoolib.programs.archmage',
                             'patoolib.programs.arj',
                             'patoolib.programs.bsdcpio',
                             'patoolib.programs.bsdtar',
                             'patoolib.programs.bzip2',
                             'patoolib.programs.cabextract',
                             'patoolib.programs.chmlib',
                             'patoolib.programs.clzip',
                             'patoolib.programs.compress',
                             'patoolib.programs.cpio',
                             'patoolib.programs.dpkg',
                             'patoolib.programs.flac',
                             'patoolib.programs.genisoimage',
                             'patoolib.programs.gzip',
                             'patoolib.programs.isoinfo',
                             'patoolib.programs.lbzip2',
                             'patoolib.programs.lcab',
                             'patoolib.programs.lha',
                             'patoolib.programs.lhasa',
                             'patoolib.programs.lrzip',
                             'patoolib.programs.lzip',
                             'patoolib.programs.lzma',
                             'patoolib.programs.lzop',
                             'patoolib.programs.mac',
                             'patoolib.programs.nomarch',
                             'patoolib.programs.p7azip',
                             'patoolib.programs.p7rzip',
                             'patoolib.programs.p7zip',
                             'patoolib.programs.pbzip2',
                             'patoolib.programs.pdlzip',
                             'patoolib.programs.pigz',
                             'patoolib.programs.plzip',
                             'patoolib.programs.py_bz2',
                             'patoolib.programs.py_echo',
                             'patoolib.programs.py_gzip',
                             'patoolib.programs.py_lzma',
                             'patoolib.programs.py_tarfile',
                             'patoolib.programs.py_zipfile',
                             'patoolib.programs.rar',
                             'patoolib.programs.rpm',
                             'patoolib.programs.rpm2cpio',
                             'patoolib.programs.rzip',
                             'patoolib.programs.shar',
                             'patoolib.programs.shorten',
                             'patoolib.programs.star',
                             'patoolib.programs.tar',
                             'patoolib.programs.unace',
                             'patoolib.programs.unadf',
                             'patoolib.programs.unalz',
                             'patoolib.programs.uncompress',
                             'patoolib.programs.unrar',
                             'patoolib.programs.unshar',
                             'patoolib.programs.unzip',
                             'patoolib.programs.xdms',
                             'patoolib.programs.xz',
                             'patoolib.programs.zip',
                             'patoolib.programs.zoo',
                             'patoolib.programs.zopfli',
                             'patoolib.programs.zpaq'],
             # hiddenimports=['Archive', 'PatoolError'],
             hookspath=[],
             runtime_hooks=[],
             excludes=[],
             win_no_prefer_redirects=False,
             win_private_assemblies=False,
             cipher=block_cipher,
             noarchive=False)
pyz = PYZ(a.pure, a.zipped_data,
             cipher=block_cipher)
exe = EXE(pyz,
          a.scripts,
          [],
          exclude_binaries=True,
          name='test',
          debug=False,
          bootloader_ignore_signals=False,
          strip=False,
          upx=True,
          console=True )
coll = COLLECT(exe,
               a.binaries,
               a.zipfiles,
               a.datas,
               strip=False,
               upx=True,
               upx_exclude=[],
               name='test')

and then after an unexpected long time, I get the following:

0
a
C:\Users\myname\eclipse-workspace-tms\test_unzip_exe\dist\test
b
c
d
e
1
2

where the file in the destination ("grog") is not an unzipped file as wanted but simply a copy.

Does anybody have an idea of what is going wrong?

Thanks a lot

EDIT:

I have made some progress: In the script, after print(1), if I add:

sys.executable = "C:\\Users\\myname\\AppData\\Local\\Continuum\\anaconda3\\python.exe"

Then it works again. However I have a non portable .exe

Since importing only the python.exe file via the spec file is not enough, the other solution would be to add into the spec file

("C:\\Users\\myname\\AppData\\Local\\Continuum\\anaconda3\*,'.'")

which is going to make a huge executable, or pick one by one only the necessary files through trial and error, which takes ages. Any elegant solution is welcome.

EDIT 2: For more insight, the part from the pyunpack where the bug occurs is at the init file located in:

"C:\\Users\\myname\\AppData\\Local\\Continuum\\anaconda3\\Lib\\site-packages\\pyunpack\\__init__.py"

in the function extract_all_patool:

   `p = EasyProcess([
        sys.executable,
        patool_path,
        '--non-interactive',
        'extract',
        self.filename,
        '--outdir=' + directory
        # '--verbose'
    ]).call(timeout=self.timeout) `

the problem being that sys.executable is set to the test.exe file instead of the python.exe executable itself

EDIT3: I found a semi-portable solution, not ideal but I found nothing better yet:

The end-user is expected to install anaconda in the default path, then pip install patool and pip install pyunpack, and then copy the exe file anywhere in his username folder.

On my side: I am adding in the spec file import pathlib, from pathlib import Path In the hidden_mports list of the spec file, I am adding: 'pathlib', 'pathlib.Path',

Then in the code, after print(1), I am adding:

abspath = abspath(".")
user_path = Path(abspath).parts
user_path = join(user_path[0], user_path[1], user_path[2], user_path[3] )
conda_path = join("AppData", "Local", "Continuum", "anaconda3", "python.exe")
sys.executable = join(user_path, conda_path)

Answer 1

The problem is, as you found in your 2nd edit, that pyunpack needs patool installed on the target system - which, in turn, requires python.

To fix this you need to:

• Use tarballs or zipfiles; shutil supports theses.

• Or require the end user to have python installed.

Another way of putting it is that to use pyunpack with pyinstaller, you need python on the target system.