Point to point messaging in scalabale application?

Question

After googling how message is sent/received in chat messenger like whatsapp, i came across they use queues based messaging system. I am just trying to figure out what can be high level design of this feature

HLD per mine understanding :- Say Friend 1 and Friend 2 are online . Friend 1 has established HTTP web connection to web server 1 and Friend 2 has established HTTP web connection to web server 2. Friend 1 send the message to Friend 2.

Now as soon as message comes to web server 1, i need to convey the message to web server 2 so that message can be pushed back to friend 2 through already established web connection.

I believe distributed custom java queues can be used here to propagate the message from one server to another. As soon as message comes to one server , it will push it to distributed queue(distribute queue because of load balancing and high availability) with message content, fromUserId, toUserId. There will be listener on queue which will see destination userId of just poppedIn message and find on which webserver destination userId is active . If user is active pop out the message and push it to client otherwise store it in db so that it can be pulled once once gets online. To see which user is active on which server, there we can maintain the treemap with userId as key and value as serverName for efficient look up

Probably actual design must be more complex/scalable than above brief . Would like to know if this is the right direction for scalable chat messenger?

Also i believe we need to have multiple distributed queues instead of one for such a scalable application. But if we have multiple distributed queues how system will ensure the FIFO message delivery across distributed queues ?

Answer 1

Would like to know if this is the right direction for scalable chat messenger?

Designing this application using message queues has the following benefits:

• Decoupling of client-server and reduce of failure blast: Queues can gracefully handle traffic peaks, by just having a temporarily increased queue size, which will be back to normal as long as traffic normal again (or any transient failures have been fixed)
• In a messaging application, clients (mobiles) can be offline for long periods. As a result, a synchronous design would not work, since the clients might not be accessible for message delivery. However, with an asynchronous design as with message queues, the responsibility of message delivery is on the client side. As a result, the client can poll for new messages as soon as it gets online.

So, yes this design could be quite scalable in terms of performance and usability. The only thing to have in mind is that this design would require a separate queue for each user, so the number of queues would scale linearly with the number of the application's users (which could be a significant financial & scalability issue).

But if we have multiple distributed queues how system will ensure the FIFO message delivery across distributed queues ?

Many queues, either open-source (rabbitMQ, activeMQ) or commercial (AWS SQS), support FIFO ordering. However, the FIFO guarantee inside the queue is not enough, since the messages sent by a single client could be delivered to the queue in different order due to asynchronicity issues in the network (unless you are using a single, not-distributed queue and TCP which guarantees ordered delivery).

However, you could implement FIFO ordering on the client side. Following this approach, the messages would include a timestamp, which would be used by each client to sort the messages when receiving them. The only side-effect of that is that a client could see a message, without having seen all the previous messages first. However, when previous messages arive, they will be shown in the correct order in the client's UI, so eventually the user would see all the messages and in the correct order.

Answer 2

 Would like to know if this is the right direction for scalable chat messenger?

I would probably prefer a slightly different approach. Your ideas are correct, but I would like to add up a bit more to the same. I happened to create such a chat messenger a few years ago, and it was supposed to be quite similar to watsapp. I am sure that when you googled, you would have come across XMPP Extensible messaging and presence protocol. we were using openfire as the server that maintains connections . The concept that you explained where

Say Friend 1 and Friend 2 are online . Friend 1 has established HTTP web connection to web server 1 and Friend 2 has established HTTP web connection to web server 2. Friend 1 send the message to Friend 2.

is called federation, and openfire can be run in a federated mode. After reading through your comments, i came across the one queue per user point. I am sure that you already know that this approach is not scalable as its very resource intensive. A good approach would be use an Actor framework such as akka. Each actor is like a light weight thread in java and each actor has an inbox. so messaging is taken care of in this case.

So your scenario transforms to Friend 1 opens a connection to openfire xmpp server and initializes a Friend 1 Actor.When he types a message, it is transferred to the Friend 1 actor's in-box ( Each actor in akka has an in memory inbox). This is communicated to the xmpp server. The server has a database of its own, and since it is federated with other xmpp servers, it will try to find if friend 2 is online. The xmpp server will keep the message in its db until the friend 2 comes online. Once friend 2 establishes a connection to any of the xmpp server a friend 2 actor is created and its presence is propagated to all other servers and the xmpp server 1 will notify Friend 2's actor. Friend 2's actor inbox will now get the message

Optional: There is also a option of delivery receipt. Once Friend2 reads the message, a delivery receipt can be sent to friend 1 to indicate the status of the message i.e read, unread, delivered, not delivered etc.