Use of messaging like RabbitMQ in web application?

Question

I would like to learn what are the scenarios/usecases/ where messaging like RabbitMQ can help consumer web applications.

Are there any specific resources to learn from?

What web applications currently are making use of such messaging schemes and how?

Answer 1

In general, a message bus (such as RabbitMQ, but not limited to) allows for a reliable queue of job processing.

What this means to you in terms of a web application is the ability to scale your app as demand grows and to keep your UI quick and responsive.

Instead of forcing the user to wait while a job is processed they can request a job to be processed (for example, clicking a button on a web page to begin transcoding a video file on your server) which sends a message to your bus, let's the backend service pick it up when it's turn in the queue comes up, and maybe notify the user that work has/will begin. You can then return control to the UI, so the user can continue working with the application.

In this situation, your web interface does zero heavy lifting, instead just giving the user visibility into stages of the process as you see fit (for example, the job could incrementally update database records with the state of process which you can query and display to your user).

I would assume that any web application that experiences any kind of considerable traffic would have this type of infrastructure. While there are downsides (network glitches could potentially disrupt message delivery, more complex infrastructure, etc.) the advantages of scaling your backend become increasingly evident. If you're using cloud services this type of infrastructure makes it trivial to add additional message handlers to process your jobs by subscribing to the job queue and just picking off messages to process.

Answer 2

I just did a Google search and came up with the following:

• Reddit.com
• Digg.com
• Poppen.De

That should get you started, at least.