Client/Server state synchronization for desktop application

Question

I am working on a desktop application that requires synchronization between several clients. Basically, a group of people (let's say between 2 and 10) all run the same application. One of them hosts a server and the other clients connect to that server. The client that hosts the server also connects to his own server.

The applications should stay synchronized between all clients, meaning all clients see the same data in the application. Specifically, the data in question I can define in two separate forms:

1. A simple property with a certain value (this value must stay synchronized)
2. A list of properties (the items in the list and their values must stay synchronized)

Simple examples of (1) could be: which item in a list does the client currently have selected, and what's the current location of the client's mouse pointer within the application window. These properties keep changing continuously but the number of these properties is constant and does not grow (e.g. defined during design time).

An example of (2) could be a list of chat messages. These lists will grow during runtime with no way to predict how many items there will be.

Here is an example code in C# for the state, client and chat messages:

public class State
{
    // A single value shared between all clients
    public int SimpleInteger {get;set;}

    // List of connected clients and their individual states
    public List<Client> Clients {get;set;}

    // List of chat messages
    public List<ChatMessage> Messages {get;set;}
}

public class Client 
{
    public string ClientId {get;set;}
    public string Username {get;set;}

    public ClientState ClientState {get;set;}
}

public class ClientState 
{
    public string ClientId {get;set;}
    public int SelectedIndex {get;set;}
    public int MouseX {get;set;}
    public int MouseY {get;set;}
}

public class ChatMessage 
{
    public string ClientId {get;set;}
    public string Message {get;set;}
}

I've been working on this on and off for a long time but whatever kind of state synchronization I came up with, it never worked well.

When I search for solutions, I only ever find solutions for games, but those are not very helpful because my requirements are different:

• I cannot deal with "dropped updates", I cannot predict (interpolate or extrapolate) what the other clients are doing. Every client needs to receive every update to stay in sync.
• On the other hand, I don't care about lag (within reason). It is fine if I see the updates of other client with about a second delay.
• When a new client connects (or reconnects), a large portion of the state must be transfered (for example: the list of chat messages from example 2). Each client is required to know about the entire history of the chat so this must be downloaded when a client connects.

My current solution can be summarized as follows:

• The server keeps track of the state, e.g. the source of truth.
• The state contains the properties that require synchronizing.
• The state also contains a list of connected users (and their usernames etc).
• Clients also each keep a local copy of the state, which they can act upon immediately. For example, they update their mouse position in their local state continously.
• Whenever a client updates his local state, this update is sent to the server.
  • Potential exceptions here are things that change too fast such as the mouse position, those I will only send in regular intervals.
• The server also updates the common "source of truth" state.
• Finally, the server updates all other clients with the new updated state.

The last two steps are where I'm struggling. I can think of two methods to synchronize the state, one is easy but probably not efficient and the other is efficient but prone to errors.

1. The server simply sends the entire state to all clients.
   • As soon as the server receives an update from the client, the update is applied to the state and the new state is broadcasted. Every other client replaces their local state.
   • I feel this will probably work, but the state can grow in size quickly due to the "list" items (for example chat messages). In my previous attempts, this quickly became a problem and sending the state back become much too slow.
2. The server re-sends the same update (that it received) to all other clients.
   • Each client then only applies the new update to their state locally to sync back with the server.
   • This is probably much more efficient and sending the entire state is only necessary when a client connects.
   • However, in the past I frequently ran into desync issues where clients were no longer in sync. I don't really know what caused it, probably conflicts between messages (for example server telling the client to update a value in the state, but the client just updated his local value, which has precedence?). After this happens, everything went completely wrong as the updates are now being applied to two different states and have different outcomes.

I'm looking for some guidance on general concepts on how to achieve this. I'm using several messaging libraries to achieve the actual communication between client and server and that part is not an issue I think. I can make sure in these libraries that every message is received for example (though I'm not sure if the order is guaranteed). Like I said before, lag is not an issue, but I must guarantee every state update is received both by the server and by every other client.

Any help would be great! Thanks.

Answer 1

This is a hard problem and there are enough tricky areas that I wouldn't want to build this myself. Authentication, conflicting updates, API management, network outages, single point of failure, and local persistence come to mind.

If you're up for using a cloud-based solution, Google Cloud Firestore takes care of those tricky areas and does what you need:

• Clients save data to the database, by creating, updating, or deleting records. Example code.
• Whenever a record is created, updated, or deleted, all clients get realtime notifications. Example code.

(After you follow the links above, make sure you click C# above the code boxes to see the C# code).