Make Python RETURN 200 Response and then run Code

Question

I'm running an AWS Lambda function that is giving me a 504 time response error.

I would like to return a 200 return status first, and then allow the Lambda function to continue to do its time consuming code.

The Lambda function performs perfectly despite the time error so I would like to kill this error.

On API Gateway I found its response has to come back within 290000 ms. So this is the workaround I'm thinking.

import json
import boto3
import subprocess
import os
import time


def lambda_handler(event, context):
    # Trigger some really time consuming code <---- RUN IT AFTER RETURNING 

    return {  #         <---- RETURN THIS RIGHT AWAY 
        'statusCode': 200,
        'body': json.dumps('Hello from Lambda! Real Test')
    }

Answer 1

You should probably return a 202 (accepted) rather than a 200. You don't know if the result will be successful, you just know that you took the work to do and spun it off on another thread.

        LhasaDad commented 1 hour ago.

 

A possible option would be to spin off a thread to do the long running item and return a 202 (accepted) to indicate to the caller that the work to do was accepted.

[...]

        LhasaDad answered 1 hour ago.

 
These above are fundamentally right, but I think lack detail somewhat important in this vendor's solutions.

The AWS way to do this is:

1. "AWS Lambda device 1" servicing AWS API Gateway approves arguments (or not), if arguments are right, fails to fetch results (see the rest of steps), continues to step 2, if arguments were wrong returns code in range 4xx.

2. "Lambda device 1" emits a message to an AWS SQS queue, if successful, returns the 202 code, else returns code in range 5xx.

3. "Lambda device 2", triggered by an SQS event, processes the SQS message, dumps answer to some other SQS queue.

4. Next time 1. is called, succeeds to fetch results, returns success 200.

Alternatively to SQS, can use AWS S3 as shared storage, but that requires creating an implicit protocol, so being more artisan and so more prone to protocol issues supposedly absent from SQS.

Either way you have to beware the possibility of creating several repeated tasks. A second call to the API should not immediately create the task if the result is not available, but first check if a task has not been recently created by another recent call passing the same parameters.

Answer 2

You can try to call your time consuming code in separate function asynchronously using threads You do not have to wait until the task is done before return

    ...
    import time
    import json
    from threading import Thread


    def time_consuming_code(arg1, arg2):
        time.sleep(5)
        print("done")


    def lambda_handler(event, context):
        # TODO implement
        thr = Thread(target=time_consuming_code, args=("p1", "test"))
        thr.start()
        return {         # <---- RETURN THIS RIGHT AWAY
            'statusCode': 200,
            'body': json.dumps('Hello from Lambda! Real Test')
        }
    print(lambda_handler("test", "test"))