Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

AWS Lambda and Python's .pyc files

Tags:

I'm trying to understand the exact effect of including .pyc files with a Python package to AWS Lambda.

The very few references I was able to find on this say there is no need to include .pyc files with the package. However, I'm seeing huge performance hit with my Lambda function when I don't include these files.

Trying to include a library with my package (Jinja2, for example), when omitting its .pyc files, the time it takes for the import jinja2 is always over 3 seconds.

When I do provide .pyc files, the first execution still takes 3 seconds but after that, it goes down to 100-200ms (I guess until the function gets eventually unloaded?).

I found this SO question which may suggest that AWS Lambda is unable to save its own compiled files, does this make sense?

My questions are - is there any definite source of information regarding the usage .pyc files with Python on AWS Lambda? Is there any way to make AWS Lambda save its own .pyc files? Or should I just continue to include them with my package?

like image 298
danielv Avatar asked Aug 13 '17 19:08

danielv


People also ask

Can I use Python packages with AWS Lambda?

The packages should have been installed along with their dependencies in the python folder (or your specific folder). You can now zip up that folder as python. zip and exit the container. You will need to copy the zipped folder into your local environment so you can upload it to the Lambda Layer or S3.

What is the difference between .py and .PYC files?

py files contain the source code of a program. Whereas, . pyc file contains the bytecode of your program.

What is the use of .PYC file in Python?

pyc files are created by the Python interpreter when a . py file is imported. They contain the "compiled bytecode" of the imported module/program so that the "translation" from source code to bytecode (which only needs to be done once) can be skipped on subsequent imports if the . pyc is newer than the corresponding .


1 Answers

I don't think so, the .pyc files is a Byte code which Python interpreter compiles the source to and then this code is executed. With this in mind, we can control what files should the "python virtual machine runs".

I Think the best solution is like you said:

just continue to include them with my package.

Once you do that, the code executor already have your bytecode files.

like image 130
Ederson Badeca Avatar answered Oct 01 '22 04:10

Ederson Badeca