perl: how I can deal with multiple sockets using select AND fork?

Question

I'm working in a small project where I need to read something from one source (file, socket, etc) ,process and write in many clients (and it is different for each new client).

I'm using select to multiplexing the I/O.

If I can_read the source, I store in an internal buffer, process and write in many output buffers (one per client). If I can_write I just write the buffer. Sounds good.

But How I can scale this for a great number of users? I was thinking in using fork and set something like x connections per process. I really don't know if it is a good way or it is better preforking or work with different ports and use some load balance BUT right now it is not clear to me how can I deal with fork + select

for example, if I work with 100 clients per process, in the 101th cliente I will fork but I still have the old select and 100 sockets clients in memory. I can clean the "old" select and start a new one but it is strange. I don't know if my strategy is good enought.

Answer 1

Usually this is made with "prefork" technique. This works like that:

1) Create listening socket

2) fork() to make N workers

3) Each worker begin accept() clients (most free worker will take the job); Meanwhile "parent" process becomes "manager" and waitpid() his childs, forking if number of "workers" < N

But this way you cannot send data between "workers" (as in your initial case with 101th client in other process), so maybe this design not fit your needs. In this case i'd prefer to use AnyEvent CPAN module. This enables you asyncronous working with way more clients at a time, since it uses epoll/kqueue for multiplexing and its scalable to thousands connects on one signle process (if your script isnot CPU-heavy).