Scaling Out Socket Servers

Question

Assuming live chat clients (Skype, Windows Live Messenger) use sockets to stay connected to their relative services, what are some strategies that the developers implement to scale out their servers? Even a system like Xbox LIVE where users are able to chat and send out game invites to their online friends.

The main problem is that each of these connections have to share state; some of this state needs to be queried by other clients (who could be connected to a different server behind a load balancer on the other side of the world). The most obvious one is online status.

Do these services use giant RAM based caches (maybe something like memcached) or NoSQL databases (like Cassandra) which all servers around the world connect to and update and retrieve the required state information.

I was wondering if this sort of solution would be fast (or reasonable) enough for real time services like the ones I described above.

My main problem is with memory. Distributing load is fairly straight forward (i hope) with a combination of load balancers and round-robin DNS balancing.

Answer 1

Here is one way. Though not necessarily concerned with memory based caching