Python bytecode (.pyc) files have a header that starts with a magic number that changes between Python versions. How can I (programmatically) find out that number for the current Python version in order to generate a valid header? I'm currently hard-coding the one for Python 3.7.1, but that means I now depend on a specific Python version.
This answer does exactly what I want using py_compile.MAGIC
, but that does not seem to exist anymore in Python 3. How can I do the equivalent in Python 3?
Here's an example of what I'm trying to do:
import dis
import marshal
PYC_HEADER = b'\x42\x0d\x0d\x0a\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
def f():
print('Hello World')
with open('test.pyc', 'wb') as pyc:
pyc.write(PYC_HEADER)
marshal.dump(dis.Bytecode(f).codeobj, pyc)
This should create a file test.pyc
, which can then be run, using the same Python interpreter as the script, and should print "Hello World!". And it does, but only when using Python 3.7. I'm looking for a way that generates the header for whichever version of Python 3 is used to run the script, rather than hard-coding 3.7.
For context:
I'm compiling a simple programming language to different bytecode formats (LLVM, Java bytecode, Web Assembly and now Python bytecode) as part of a planned tutorial series on compiler construction.
I can generate the Python bytecode using the byteasm library, which gives me a function as a result. But in order to write the contents to a .pyc
file, I need a valid header. By hard-coding the header, the code will only work if the people following the tutorial are running the same version of Python 3 as I am (3.7) or they'd have to manually find out the magic number for their version.
Steps to Check Magic Number in Python Take a number as input (num). Calculate the sum of digits of the input number. Repeat step 2 until we get a single digit. If resultant sum is equal to 1 then it is a Magic number else not.
The "magic number" comes from UNIX-type systems where the first few bytes of a file held a marker indicating the file type. Python puts a similar marker into its pyc files when it creates them. The Python interpreter ensures that this number is correct when loading the file.
pyc files are created by the Python interpreter when a . py file is imported. They contain the "compiled bytecode" of the imported module/program so that the "translation" from source code to bytecode (which only needs to be done once) can be skipped on subsequent imports if the . pyc is newer than the corresponding .
As of Python 3.4 there is the importlib.util.MAGIC_NUMBER
in the module importlib
:
>>> import importlib
>>> importlib.util.MAGIC_NUMBER.hex()
'420d0d0a'
Another solution for Python < 3.4 or Python2 is the get_magic
method of the imp
module.
>>> import imp
>>> imp.get_magic().hex()
'420d0d0a'
Note, that while this still works in Python 3.7, it is deprecated since Python 3.4
It moved!1
>>> import importlib.util
>>> importlib.util.MAGIC_NUMBER
b'3\r\r\n'
New in version 3.4.
Probably simplest to wrap this in a try/except to fall back to py_compile
.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With