Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Does this web app require a task queue?

Background

I have a web app that will create an image from user input. The image creation could take up to a couple seconds.

Problem

If I let the server thread, that is handling the request/response also generate the image, that is going to tie up a thread for a couple seconds, and possibly bog down my server, affect performance, kill puppies, etc.

Question

Should I use a task queue, such as Celery, so that the server can hand off the image creation, and go back to handling requests/responses? I have no problem letting the user who is creating the image wait, but I dont want it to effect other peoples access to the site.

like image 695
random_person Avatar asked Aug 26 '10 18:08

random_person


People also ask

Why do we need task queue?

Task queues let applications perform work, called tasks, asynchronously outside of a user request. If an app needs to execute work in the background, it adds tasks to task queues. The tasks are executed later, by worker services. The Task Queue service is designed for asynchronous work.

What is Google task queue?

Task queue functions take advantage of Google Cloud Tasks to help your app run time-consuming, resource-intensive, or bandwidth-limited tasks asynchronously, outside your main application flow.

What is queue in Web server?

The Web-Queue-Worker architecture defines a web portion that handles HTTP requests and a worker portion that handles time or processing-intensive operations. A queue is used for asynchronous communication between the web and the worker.


1 Answers

I'm going to say No - for now.

  • A couple of second is not that long.
  • You'll anyway have to implement some sort of polling (or comet processing) to feed the image back to the user.
  • It will make your system more complex.
  • Design the system so adding on a task queue later on is feasible and easy.

So, keep it simple at first and get it working, but Keep in mind that you might add a task queue later.

Implement that task queue when/if you need to scale.

like image 89
Anonym Avatar answered Nov 08 '22 12:11

Anonym