Menu
I am currently trying to implement a job queue in php. The queue will then be processed as a batch job and should be able to process some jobs in parallel.
I already did some research and found several ways to implement it, but I am not really aware of their advantages and disadvantages.
E.g. doing the parallel processing by calling a script several times through fsockopen like explained here:
Easy parallel processing in PHP
Another way I found was using the curl_multi functions.
curl_multi_exec PHP docs
But I think those 2 ways will add pretty much overhead for creating batch processing on a queue that should mainly run on the background?
I also read about pcntl_fork which also seems to be a way to handle the problem. But that looks like it can get really messy if you don't really know what you are doing (like me at the moment).
I also had a look at Gearman, but there I would also need to spawn the worker threads dynamically as needed and not just run a few and let the gearman job server then sent it to the free workers. Especially because the threads should be exit cleanly after one job has been executed, to not run into eventual memory leaks (code may not be perfect in that issue).
Gearman Getting Started
So my question is, how do you handle parallel processing in PHP? And why do you choose your method, which advantages/disadvantages may the different methods have?
205
Parallel processing involves taking a large task, dividing it into several smaller tasks, and then working on each of those smaller tasks simultaneously. The goal of this divide-and-conquer approach is to complete the larger task in less time than it would have taken to do it in one large chunk.
Multi-threading is possible in php. From the PHP documentation: pthreads is an object-orientated API that provides all of the tools needed for multi-threading in PHP. PHP applications can create, read, write, execute and synchronize with Threads, Workers and Threaded objects.
In parallel processing, we take in multiple different forms of information at the same time. This is especially important in vision. For example, when you see a bus coming towards you, you see its color, shape, depth, and motion all at once. If you had to assess those things one at a time, it would take far too long.
i use exec(). Its easy and clean. You basically need to build a thread manager, and thread scripts, that will do what you need.
I dont like fsockopen() because it will open a server connection, that will build up and may hit the apache's connection limit
I dont like curl functions for the same reason
I dont like pnctl because it needs the pnctl extension available, and you have to keep track of parent/child relations.
never played with gearman...
122
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
