Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Can you have an async handler in Lambda Python 3.6?

I've made Lambda functions before but not in Python. I know in Javascript Lambda supports the handler function being asynchronous, but I get an error if I try it in Python.

Here is the code I am trying to test:

async def handler(event, context):
    print(str(event))
    return { 
        'message' : 'OK'
    }

And this is the error I get:

An error occurred during JSON serialization of response: <coroutine object handler at 0x7f63a2d20308> is not JSON serializable
Traceback (most recent call last):
  File "/var/lang/lib/python3.6/json/__init__.py", line 238, in dumps
    **kw).encode(obj)
  File "/var/lang/lib/python3.6/json/encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/var/lang/lib/python3.6/json/encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
  File "/var/runtime/awslambda/bootstrap.py", line 149, in decimal_serializer
    raise TypeError(repr(o) + " is not JSON serializable")
TypeError: <coroutine object handler at 0x7f63a2d20308> is not JSON serializable

/var/runtime/awslambda/bootstrap.py:312: RuntimeWarning: coroutine 'handler' was never awaited
  errortype, result, fatal = report_fault(invokeid, e)

EDIT 2021:

Since this question seems to be gaining traction, I assume people are coming here trying to figure out how to get async to work with AWS Lambda as I was. The bad news is that even now more than a year later, there still isn't any support by AWS to have an asynchronous handler in a Python-based Lambda function. (I have no idea why, as NodeJS-based Lambda functions can handle it perfectly fine.)

The good news is that since Python 3.7, there is a simple workaround in the form of asyncio.run:

import asyncio

def lambda_handler(event, context):
    # Use asyncio.run to synchronously "await" an async function
    result = asyncio.run(async_handler(event, context))
    
    return {
        'statusCode': 200,
        'body': result
    }

async def async_handler(event, context):
    # Put your asynchronous code here
    await asyncio.sleep(1)
    
    return 'Success'

Note: The selected answer says that using asyncio.run is not the proper way of starting an asynchronous task in Lambda. In general, they are correct because if some other resource in your Lambda code creates an event loop (a database/HTTP client, etc.), it's wasteful to create another loop and it's better to operate on the existing loop using asyncio.get_event_loop.

However, if an event loop does not yet exist when your code begins running, asyncio.run becomes the only (simple) course of action.

like image 491
Abion47 Avatar asked Feb 28 '20 16:02

Abion47


People also ask

Can Lambda be async?

For asynchronous invocation, Lambda places the event in a queue and returns a success response without additional information. A separate process reads events from the queue and sends them to your function. To invoke a function asynchronously, set the invocation type parameter to Event .

Does Python support asynchronous?

Python's asyncio package (introduced in Python 3.4) and its two keywords, async and await , serve different purposes but come together to help you declare, build, execute, and manage asynchronous code.

What is Lambda handler in Python?

The Lambda function handler is the method in your function code that processes events. When your function is invoked, Lambda runs the handler method. When the handler exits or returns a response, it becomes available to handle another event.


1 Answers

Not at all. Async Python handlers are not supported by AWS Lambda.

If you need to use async/await functionality in your AWS Lambda, you have to define an async function in your code (either in Lambda files or a Lambda Layer) and call asyncio.get_event_loop().run_until_complete(your_async_handler()) inside your sync handler.

Please note that asyncio.run (introduced in Python 3.7) is not a proper way to call an async handler in AWS Lambda execution environment since Lambda tries to reuse the execution context for subsequent invocations. The problem here is that asyncio.run creates a new EventLoop and closes the previous one. If you have opened any resources or created coroutines attached to the closed EventLoop from previous Lambda invocation you will get «Event loop closed» error. asyncio.get_event_loop().run_until_complete allows you to reuse the same loop. See related StackOverflow question.

AWS Lambda documentation misleads its readers a little by introducing synchronous and asynchronous invocations. Do not mix it up with sync/async Python functions. Synchronous refers to invoking AWS Lambda with further waiting for the result (blocking operation). The function is called immediately and you get the response as soon as possible. Whereas using an asynchronous invocation you ask Lambda to schedule the function execution and do not wait for the response at all. When the time comes, Lambda still will call the handler function synchronously.

like image 173
Anton Bryzgalov Avatar answered Oct 19 '22 09:10

Anton Bryzgalov