how Message Queue System Works?

Question

I have studied Message Queues System in my class but I still don't get it how these Message Queues System work in real time scenarios? Is there any tutorial which can help me to get the complete picture? Can someone explain me how these systems work?

Answer 1

An example: My thread or process can send a message to your message queue, and having sent it, my code goes on to do something else. Your code, when it gets around to it, reads the next message from the message queue, and then decides what to do about that message. Message queues avoid needing to have a critical section or mutex shared between the two threads, or processes. The underlying message queue layer itself takes care of making sure that messages get into the queue without race conditions affecting the integrity of the queue.

Message queues can be used for both one-way and two-way, asynchronous messaging. For one-way use, my thread can use it to keep your thread appraised of key events in my thread, without acknowledgement back from your thread. For two-way use, after my thread sends a message to your thread, your thread may need to send data back to my thread via my message queue.

The message queue layer uses lower level synchronization schemes to insure that no two writers to the queue can write at the same time. It insures that all writes to the queue are atomic. It also insures that a reader of the queue cannot read a partially written message from the queue.

Most message queue APIs also offer support for reading messages from the queue based on a filter that you designate. Say for instance that you consider messages from a time critical thread to be more important that other messages. You can each time you check your queue for messages, first check for messages from the critical thread, and service those messages first. Your thread would then go onto to process the rest of the messages as normal, provided no more messages from the critical thread are found.

A C tutorial of the UNIX message queues

Answer 2

That's a complex topic but to put it simply:

Message Queues are one of the best ways, if not the best, to implement distributed systems.

Now you might ask, what is a distributed system? It is an integrated system that spans multiple machines, clients or nodes which execute their tasks in parallel in a non-disruptive way. A distributed system should be robust enough to continue to operate when one or more nodes fail, stop working, lag or are taken down for maintenance.

Then you might ask, what is a message queue? It is a message-oriented middleware that enables the development of a distributed system by using asynchronous messages for inter-node communication through the network.

And finally you might ask, what is all that good for? This is good for implementing applications with a lot of moving parts called nodes which needs real-time monitoring and real-time reaction capabilities. To summarize they provide: parallelism (nodes can truly run in parallel), tight integration (all nodes see the same messages in the same order), decoupling (nodes can evolve independently), failover/redundancy (when a node fails, another one can be running and building state to take over immediately), scalability/load balancing (just add more nodes), elasticity (nodes can lag during activity peaks without affecting the system as a whole) and resiliency (nodes can fail / stop working without taking the whole system down).

Check this article which discusses a message queue infrastructure in detail.