Delphi 7 Indy 9 multithreaded HTTP Server

Question

I need to write a multithreaded HTTP server. I know Indy handles the threading via IdThreadMgrPool.

My requirements are straightforward: have a server that accepts multiple simultaneous clients (web browser POST requests), runs a limited number of threads that call a Delphi DLL (likely separate to the connection threads), and returns the results.

Non-blocking would be the ultimate (like node.js).

There are several SO posts on this topic. What I can't find here (or via Google) is an example of how to use IdThreadMgrPool. I see posts that say it must be used, but no example how. There is no example on Indy's website either.

Could someone point me to an example? Or is there a FOS framework that will work for this?

One idea is to have Indy spawn multiple threads as normal, then to have these threads access a controlled pool of threads that run the DLL. If this is plausible, then I just need to know the "normal" case usage example.

Answer 1

To use TIdThreadMgrPool, at a minimum all you have to do is create an instance of it, assign that to the TIdHTTPServer.ThreadMgr property, and set its PoolSize property. All of that can be done at design-time.

Keep in mind that the PoolSize does not limit the number of connections on the server. The server has its own MaxConnections property for that purpose. For example, you could have a PoolSize of 10 and have 15 clients connected simultaneously, and thus 15 threads running. When they disconnect, 10 threads will be put back in the pool and 5 threads will be terminated.

To customize the pool threads, you can derive a new class from TIdPeerThread, optionally override its virtual BeforeExecute() and AfterExecute() methods to perform per-thread initializations the cleanups, and then assign that class to the server's (not the ThreadMgr's) ThreadClass property at runtime before activating the server. Inside your server event handlers, you can then typecast the provided TIdPeerThread object to your custom class and use it as needed.

You can add methods to your custom thread class and have them internally access the DLL, throttling as needed. The simplest throttle would be to use a single shared semaphore to control the number of threads that can enter the semaphore at a time. In that regard, you can then limit to, say, 2 threads at a time even if 15 threads are running.

Since you say you want to "run the DLL in a thread", a semaphore will likely not be enough. In that case, I would recommend using an I/O Completion Port instead. You can have your custom thread class post a request to the IOCP using PostQueuedCompletionStatus() and wait for the response to come back. Throttling is accomplished by the number of threads you create to service the IOCP, such as one thread per CPU core. Each IOCP thread would use GetQueuedCompletionStatus() in a loop to receive posted requests.

Indy is not asynchronous, so you would not be able to post a request to the IOCP and let it send a response back to the client directly when ready. The server sends a response back to the client using the same thread that manages the client connection. So the client thread will have to post a request to the IOCP and wait for its response, then send that response to the client. You can define a record that contains a TEvent, input values needed for calling the DLL, and output values for the DLL's response. Then create an instance of that record, post a pointer to it to the IOCP, and wait for the TEvent to be signalled. When an IOCP thread receives the record pointer, it can call the DLL as needed, fill the record with the response, and then signal the record's TEvent. The waiting client thread will then be unblocked and can send the record's response data to the client as needed.

Answer 2

Indy server does not scale well.

Consider using http.sys kernel mode server, iocp based.

Our open source SynCrtSock unit feature a high performance http server, and works perfectly from Delphi 5 up to XE6.

See for instance this sample code.