I want to run a Pythonic project using Python compilation (.pyc
or __pycache__
). In order to do that in Python2, I haven't any problem.
Here is a simplified example in a Python2 project:
Project tree:
test2
├── main.py
└── subfolder
├── __init__.py
└── sub.py
Compile:
python -m compileall test2
Project tree after the compile:
test2
├── main.py
├── main.pyc
└── subfolder
├── __init__.py
├── __init__.pyc
├── sub.py
└── sub.pyc
As you can see, several .pyc
manually generated. Now I can run this project using main.pyc
as fine, which has a relation with the sub.py
:
python main.pyc
Out:
Hi
Bye
main.py content:
from subfolder import sub
print('Bye')
sub.py content:
print('Hi')
Now I want to retry this behavior in a Python3 project.
Here is a simplified asyncio
(available in Python3) project:
Project tree:
test3
├── main.py
└── subfolder
├── __init__.py
└── sub.py
Compile:
python3 -m compileall test3
Project tree after the compile:
test3
├── main.py
├── __pycache__
│ └── main.cpython-36.pyc
└── subfolder
├── __init__.py
├── __pycache__
│ ├── __init__.cpython-36.pyc
│ └── sub.cpython-36.pyc
└── sub.py
As you can see, __pycache__
folders manually generated. But I cannot run this project using main.cpython-36.pyc
which has a relation with subfolder
:
cd test3/__pycache__
python3 main.cpython-36.pyc
Out (I expected that produced the Hi Bye
message):
Traceback (most recent call last):
File "test3/main.py", line 2, in <module>
ModuleNotFoundError: No module named 'subfolder'
main.py content:
import asyncio
from subfolder import sub
async def myCoroutine():
print("Bye")
def main():
loop = asyncio.get_event_loop()
loop.run_until_complete(myCoroutine())
loop.close()
main()
sub.py content:
print('Hi')
How can I run this project (above Python3 project) using __pycache__
folder?
Or
How can I run a Python3 project with the relation between subfolders using python compilation?
[NOTE]:
I cannot use the python compileall
(Python2 compile) in the above Python3
project due to the asyncio
method.
My Python(s) version is Python2.7 and Python3.6
You can enforce the same layout of pyc-files in the folders as in Python2 by using:
python3 -m compileall -b test3
The option -b
triggers the output of pyc
-files to their legacy-locations (i.e. the same as in Python2).
After that you can once again use the compiled cache via:
python3 main.pyc
The way the loading of modules works since PEP-3147, it is impossible to use pyc-files from __pycache__
folder in the way you intend: If there is no *.py
-file, the content of the __pycache__
is never looked-up. Here is the most important part of the workflow:
import foo
|
|
-- > [foo.py exists?] --- NO ----> [foo.pyc exists?] -- NO --> [ImportError]
| |
| YES
YES |--> [load foo.pyc]
|
|-> [look up in __pycache__]
That means, files from __pycache__
are only looked up, when a corresponding *.py
-file can be found.
Obviously, building python scripts with a Python version 3.X in this way and trying to run the resulting pyc-files with another Python version 3.Y will not work: Different Python versions need different pyc-files, this is the whole point behind PEP-3147.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With