Asynchronous task queue for serverless AWS Lambda / Django / Zappa [duplicate]

Question

I have been experimenting with deploying Django apps to AWS Lambda with Zappa.

In some of my other (EC2/EBS hosted) Django projects, if there is a need to perform some heavier calculation that can take some time (such as sending a lot of emails, or just some processing that takes over a minute), Celery is used. It is a task queue system where the tasks are sent to a queue, a response can be returned immediately and workers can process the tasks later.

What would be the best way to implement a Celery-like task queuing system for a Zappa-Django app running in Lambda?

Zappa/Lambda supports scheduled tasks, and the models of the app could be designed in such a way that the processing could be done by scheduled functions later and the results could be saved to DB. But I do not think polling for tasks once a minute is robust enough, there is a oftena need to start the delayed task immediately.

Is there an easy way to return a response from a Django view immediately and have a function (from inside the Django app) with arbitrary parameters queued to be executed later?

Answer 1

You can do it using SNS. Subscribe lambda to topic and publish messages there with json payload.