How to resuse threads - pthreads c

Question

I am programming using pthreads in C.

I have a parent thread which needs to create 4 child threads with id 0, 1, 2, 3. When the parent thread gets data, it will set split the data and assign it to 4 seperate context variables - one for each sub-thread. The sub-threads have to process this data and in the mean time the parent thread should wait on these threads. Once these sub-threads have done executing, they will set the output in their corresponding context variables and wait(for reuse). Once the parent thread knows that all these sub-threads have completed this round, it computes the global output and prints it out. Now it waits for new data(the sub-threads are not killed yet, they are just waiting).

If the parent thread gets more data the above process is repeated - albeit with the already created 4 threads.

If the parent thread receives a kill command (assume a specific kind of data), it indicates to all the sub-threads and they terminate themselves. Now the parent thread can terminate.

I am a Masters research student and I am encountering the need for the above scenario. I know that this can be done using pthread_cond_wait, pthread_Cond_signal. I have written the code but it is just running indefinitely and I cannot figure out why.

My guess is that, the way I have coded it, I have over-complicated the scenario. It will be very helpful to know how this can be implemented. If there is a need, I can post a simplified version of my code to show what I am trying to do(even though I think that my approach is flawed!)...

Can you please give me any insights into how this scenario can be implemented using pthreads?

Answer 1

As far what can be seen from your description, there seems to be nothing wrong with the principle.

What you are trying to implement is a worker pool, I guess, there should be a lot of implementations out there. If the work that your threads are doing is a substantial computation (say at least a CPU second or so) such a scheme is a complete overkill. Mondern implementations of POSIX threads are efficient enough that they support the creation of a lot of threads, really a lot, and the overhead is not prohibitive.

The only thing that would be important if you have your workers communicate through shared variables, mutexes etc (and not via the return value of the thread) is that you start your threads detached, by using the attribute parameter to pthread_create.

Once you have such an implementation for your task, measure. Only then, if your profiler tells you that you spend a substantial amount of time in the pthread routines, start thinking of implementing (or using) a worker pool to recycle your threads.