Thread Pool and Job Queue Architecture?

Question

I have an epoll to receive incoming events and put them into a Job Queue.

When the event is put into Job Queue, pthread conditional signal is sent to wake up the worker threads.

However I've met a problem where all worker threads are busy and Jobs keep stacks up on the queue. This is serious problem because if Jobs are stacked, and new event doesn't come in a while, Jobs in Queue won't be handed to Worker Threads.

As soon as Thread gets available, I wish they can take jobs from the Job Queue automatically (if Possible).

Is there anyway to do this? All I can think of is.. adding a Queue Observer and send a conditional Signal in intervals.

Also, I know that STL Queue is not thread-safe. Does that mean I have to Mutex Lock everytime when I get an access to STL Queue? WOn't this slow down my working process?

Any suggestion to solve this problem will be great.

Answer 1

The classic way of counting/signaling jobs on a producer-consumer queue is to use a semaphore. The producer signals it after pushing on a job and the consumer waits on it before popping one off. You need a mutex around the push/pop as well to protect the queue from multiple access.