Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Django - Should external API requests always be made through a task handler (e.g. Celery)?

I have a Django app where I have created a custom middleware.

It works as follows:

  1. The middleware intercepts a token (which identifies the users) within each request, and makes a request to an external API with that token.
  2. The external API returns what permissions the user making the original request has.
  3. The middleware completes, and the user gets data returned based on its permissions

This is my question: Because my app has to wait for the API request to return before it can process the request, does it still make sense to use a task queue such as celery? Wouldn't it still have to block the thread while I waiting for the response?

like image 418
andrroy Avatar asked Mar 07 '17 20:03

andrroy


People also ask

Why Celery is used in Django?

Celery makes it easier to implement the task queues for many workers in a Django application.

How does Python Celery work?

This is where Celery comes into play. Celery is a task queue implementation for Python web applications. Meaning, it allows Python applications to rapidly implement task queues for many workers. It essentially does the hard work in that it receives tasks and then assigns them to workers as needed.

How do you deploy Celery?

You deploy Celery by running one or more worker processes. These processes connect to the message broker and listen for job requests. The message broker distributes job requests at random to all listening workers.


1 Answers

No, using Celery here wouldn't make any sense at all. That's for tasks that can be purely out-of-process. A good example is sending a confirmation email; the response sent to the browser doesn't have to wait for the email to be sent, because it doesn't depend on it in any way.

In your case, the response explicitly does depend on the value from the API. There would be nothing to be gained from using Celery, and it would make the whole process much more complex than it needs to be.

like image 64
Daniel Roseman Avatar answered Oct 02 '22 23:10

Daniel Roseman