I'm trying to understand the exact effect of including .pyc files with a Python package to AWS Lambda.
The very few references I was able to find on this say there is no need to include .pyc files with the package. However, I'm seeing huge performance hit with my Lambda function when I don't include these files.
Trying to include a library with my package (Jinja2, for example), when omitting its .pyc files, the time it takes for the import jinja2
is always over 3 seconds.
When I do provide .pyc files, the first execution still takes 3 seconds but after that, it goes down to 100-200ms (I guess until the function gets eventually unloaded?).
I found this SO question which may suggest that AWS Lambda is unable to save its own compiled files, does this make sense?
My questions are - is there any definite source of information regarding the usage .pyc files with Python on AWS Lambda? Is there any way to make AWS Lambda save its own .pyc files? Or should I just continue to include them with my package?
The packages should have been installed along with their dependencies in the python folder (or your specific folder). You can now zip up that folder as python. zip and exit the container. You will need to copy the zipped folder into your local environment so you can upload it to the Lambda Layer or S3.
py files contain the source code of a program. Whereas, . pyc file contains the bytecode of your program.
pyc files are created by the Python interpreter when a . py file is imported. They contain the "compiled bytecode" of the imported module/program so that the "translation" from source code to bytecode (which only needs to be done once) can be skipped on subsequent imports if the . pyc is newer than the corresponding .
I don't think so, the .pyc
files is a Byte code which Python interpreter compiles the source to and then this code is executed. With this in mind, we can control what files should the "python virtual machine runs".
I Think the best solution is like you said:
just continue to include them with my package.
Once you do that, the code executor already have your bytecode files.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With