How to fix "Unable to import module" error in AWS lambda

Question

I wrote a script to help me in automating the creation of new micro-services. It works by taking the URI of a bitbucket repository and AWS credentials. It clones the repository, adds a bitbucket-pipelines.yml file. Creates a new lambda function by invoking the CLI and uses a template zip as the base code. It also sets the lambda handler to main.lambda_handler. It then adds the necessary repository variables to bitbucket and commits the project files. The pipeline then runs and updates the code on the lambda. Everything seems fine until I try to run the Lambda.

Here is the bitbucket-pipelines.yml file

image: peteremil/zipaws

pipelines:
  default:
    - step:
        name: Deploy code to S3
        deployment: staging
        script:
          - mkdir packages
          - pip install -r requirements.txt --target=packages
          - chmod 777 -R *
          - zip -r $semver.zip .
          - aws s3 cp $semver.zip s3://{code_s3_bucket}/$staging_location/
          - aws lambda update-function-code --region $aws_region --function-name {function_name}-function --s3-bucket {code_s3_bucket} --s3-key $staging_location/$semver.zip

Here is the main.py file

import json
import os
import sys
import requests

packages_path = os.path.join(os.path.split(__file__)[0], "packages")
sys.path.append(packages_path)


def main(local_id):
    return {
        "success" : local_id
    }


def lambda_handler(event, context):
    local_id = event["queryStringParameters"]["local_id"]
    response = main(local_id)
    return {
        "statusCode": 200,
        "headers": {"content-type": "application/json"},
        "body": json.dumps(response)
    }


if __name__ == '__main__':
    local_id = sys.argv[-1]
    response = main(local_id)
    print(response)

I created the following Test Event

{
    "queryStringParameters": {
        "local_id": "00000000"
    }
} 

The expected result of the event I created should be

{
    "statusCode": 200,
    "headers" : {"content-type": "application/json"},
    "body" : {
        {
            "success" : "00000000"
    }
}

but instead the lambda fails execution and gives me

{
    "errorMessage": "Unable to import module 'main'"
}

I tried renaming the handler, getting the settings of another working lambda (that was created manually from the console) and seeing if there are any differences but I always get the same error. The main.py file shows up fine in the inline code editor though.

Update 1 I added the import requests line to main.py because it turns out that it was causing the problem. I thought it was harmless which is why I didn't include it in the original question body.

Answer 1

As it turns out, in my code, I had an import of a library installed inside the packages\ folder but I imported it before adding the folder to sys.path so the solution was to import requests after adding the packages_path path to sys.path.

main.py now starts like this

import json
import os
import sys

packages_path = os.path.join(os.path.split(__file__)[0], "packages")
sys.path.append(packages_path)

import requests #moved this line down after adding packages_path to sys.path