What is the best, most efficient, Client pool technique with Erlang

Question

I'm a real Erlang newbie (started 1 week ago), and I'm trying to learn this language by creating a small but efficient chat server. (When I say efficient I mean I have 5 servers used to stress test this with hundreds of thousands connected client - A million would be great !)

I have find some tutorials doing so, the only thing is, that every tutorial i found, are IRC like. If one user send a message, all user except sender will receive it. I would like to change that a bit, and use one-to-one discussion.

What would be the most effective client pool for searching a connected user ? I thought about registering the process, because it seems to do everything I need, but I really don't think this is the better way to do it. (Or most pretty way to do it anyway).

Does anyone would have any suggestions doing this ?

EDIT :

Every connected client is affected to an ID.

When the user is connected, it first send a login command to give it's id. When an user wants to send a message to another one the message looks like this

[ID-NUMBER][Message] %% ID-NUMBER IS A FIXED LENGTH

When I ask for "the most effective client pool", I'm actually looking for the fastest way to retrieve/add/delete one client on the connected client list which could potentially be large (hundred of thousands -- maybe millions)

EDIT 2 :

For answering some questions :

• I'm using Raw Socket (Using telnet right now to communicate with server) - will probably move to ssl later...
• It is my own protocol
• Every Client is a spawned Pid
• Every Client's Pid is linked to it's own monitor (mostly for debugging reason - The client if disconnected should reconnect by it's own starting auth from scratch)
• I have read a couple a book before starting coding, So I do not master yet every aspect of Erlang but I'm not unaware of it, I will read more about it when needed I guess.
• What I'm really looking for is the best way to store and search thoses PIDs to send message directly from process to process.

Should I write my own search Client function using lists ?

or should I use ets ?

Or even use register/2 unregister/1 and whereis/1 to maintain my client list, using it's unique id as atom, it seems to be the simplest way to do so, I really don't know if it is efficient, but I'm pretty sure this is the ugly solution ;-) ?

Answer 1

I'm doing something similar to your chat program using gproc as a pubsub (similar to the demo on that page). Each client registers as it's id. To find a particular client, you do a lookup on that client id. To subscribe to a client, you add a property to that process of the client id being subscribed to. To publish, you call gproc:send(ClientId,Message). This covers your use case, the more general room based chat as well, and can handle distributed masterless registry of processes.

I haven't tested to see if it scales to millions, but it uses ets to do the storage and gproc is rock solid code by Ulf Wiger. I wouldn't count on being able to write a better implementation.