Creating an on-the-fly zip file from string content for AWS Lambda in Python

Question

I have a Python script that creates a Lambda script in AWS along with all the policies and triggers. I use python boto3 library for that. I create the zip file for the lambda as on-the-fly rather than uploading a static zip file from the hard drive. I use this simple code from below to create my zip file. It creates the zip file without any problems and my python code uploads this zip file as a lambda script and I can view my lambda script in the AWS without any problems. But when I run my lambda script it gives me the module not found error even though I can clearly see that both the module name and the file name does exist and is view-able.

Unable to import module 'xxxx': No module named xxxx

In the file system I double click that zip file that was created by this code and see that the content is created and everything looks normal.

If I bypass zipping on the fly and create the zip statically using WinZip and let the rest of the Python & boto3 script upload this file then it works just fine.

def CreateLambdaZip(self, fileName, fileContent):
    with zipfile.ZipFile('Lambda/' + fileName + '.zip', 'w') as myzipc:
        myzipc.writestr( fileName + '.py', fileContent)
        myzipc.close()

It kinda looks like for the zip file I'm skipping some special headers that is needed by Aws Lambda. Is there such thing? Because in the file system the zip file that is created by Python code and the other one that is created by WinZip are exactly the same. So I know there's nothing wrong with the lambda script.

Update: I'm uploading the zip file using the below code that reads the zip file which was created using the above snippet.

with open('Lambda/'+ fileName +'.zip', 'rb') as zipFile:
    func = boto3.client("Lambda").create_function(
                FunctionName=lambdaFunction,
                Runtime='python2.7',
                Role=role['Role']['Arn'],
                Handler= fileName + "." + functionName,
                Description=description,
                Timeout=10,
                MemorySize=256,
                Publish=True,
                Code={'ZipFile': zipFile.read()},
            )

When I use zipFile.read() I get 2 different headers for the same content when I zip it using WinZip and when I zip it using Python's module. Zip file that's created programmatically using Python

b'PK\x03\x04\x14\x00\x00\x00\x00\x00\xe4~\x01IO\x96J=Z\x07\x00\x00Z\x07\x00\x00\x19\x00\x00\x00schedule-ec2-snapshots.pyimport json\nimport boto3\nimport time\nfrom datetime import date, timedelta\n\nprint(\'Loading scheduled EC2 backup actions\')\n\ndef create_snapshots(event, context):\n    """\n    Lambda function that executes daily snapshots for the instances that

and zipfile created by WinZip

b'PK\x03\x04\x14\x00\x02\x00\x08\x004X\xfcH\x88\x1f\xce\xb5&\x03\x00\x00b\x07\x00\x00\x19\x00\x00\x00schedule-ec2-snapshots.py\x8dU]k\xdb@\x10|7\xf4?,\nA\x12qL\xda\x06B\r~I\x93Bh\x9b\x87&\xf4E\x15\xe1\xac[\xdb\xd7HwBw2\t\xc1\xff\xbd{+\xeb\xcb.\xb4\n\xc4\xba\xdb\xd1\xec\xce\xdc\xae\xa4\x8a\xd2T\x0e~[\xa3\'\xaa\xb9_\x1ag>\xb6\x0b\xa7\n\x9c\xac*S\x80\x14\x0e\xfd\n\xf6\x11\xbf\x9er\\b\xee\xc4dRVJ\xbb(\xfcf\x84Tz\r6\xdb\xa0\xacs\x94p\xfb\xf9\x03,E\xf6\\\x97 

Answer 1

With the info above I was able to start the in-memory solution. The deployment of that zip file worked but I could not use the resulting function. Got error:

Unable to import module '<function-name>': No module named <function-name>

I got it to work by specifying the file permissions.

I then use the in-mem-zip to create an AWS lambda function.

Setup: file_map is a dictionary of full_path->file_bytes. files is a list of full_paths

def create_lambda_function(function_name, desc, role, handler, file_map, files)
    zip_contents = create_in_mem_zip_archive(file_map, files)

    result = lambda_code.create_function(
        FunctionName=function_name,
        Runtime="python2.7",
        Description=desc,
        Role=role,
        Handler=handler,
        Code={'ZipFile': zip_contents},
    )
    return result

def create_in_mem_zip_archive(file_map, files):
    buf = io.BytesIO()
    logger.info("Building zip file: " + str(files))
    with zipfile.ZipFile(buf, 'w', zipfile.ZIP_DEFLATED) as zfh:
        for file_name in files:
            file_blob = file_map.get(file_name)
            if file_blob is None:
                logger.error("Missing file {} from files".format(file_name))
                continue
            try:
                info = zipfile.ZipInfo(file_name)
                info.date_time = time.localtime()
                info.compress_type = zipfile.ZIP_DEFLATED
                info.external_attr = 0777 << 16L  # give full access
                # info.external_attr = 0644 << 16L  # -r-wr--r--
                # info.external_attr = 0755 << 16L  # -rwxr-xr-x
                zfh.writestr(info, file_blob)
            except Exception as ex:
                logger.info("Error reading file: " + file_name + ", error: " + ex.message)
    buf.seek(0)
    return buf.read()