Async PHP | Processing data into several systems (Advice)

Question

I'm building an integration that communicates data to several different systems via API (REST). I need to process data as quickly as possible. This is a basic layout:

1. Parse and process data (probably into an array as below)

$data = array( Title => "Title", Subtitle => "Test", .....

2. Submit data into service (1) $result1 = $class1->functionservice1($data);
3. Submit data into service (2) $result2 = $class2->functionservice2($data);
4. Submit data into service (3) $result3 = $class3->functionservice3($data);
5. Report completion echo "done";

Run in a script as above I'll need to wait for each function to finish before it starts the next one (taking 3 times longer).

Is there an easy way to run each service function asynchronously but wait for all to complete before (5) reporting completion. I need to be able to extract data from each $result and return that as one post to a 4th service.

Sorry if this is an easy question - I'm a PHP novice

Many thanks, Ben

Answer 1

Yes, there are multiple ways.

The most efficient is to use an event loop that leverages non-blocking I/O to achieve concurrency and cooperative multitasking.

One such event loop implementation is Amp. There's an HTTP client that works with Amp, it's called Artax. An example is included in its README. You should have a look at how promises and coroutines work. There's Amp\wait to mix synchronous code with async code.

<?php

Amp\run(function() {
    $client = new Amp\Artax\Client;

    // Dispatch two requests at the same time
    $promises = $client->requestMulti([
        'http://www.google.com',
        'http://www.bing.com',
    ]);

    try {
        // Yield control until all requests finish
        list($google, $bing) = (yield Amp\all($promises));
        var_dump($google->getStatus(), $bing->getStatus());
    } catch (Exception $e) {
        echo $e;
    }
});

Other ways include using threads and or processes to achieve concurrency. Using multiple processes is the easiest way if you want to use your current code. However, spawning processes isn't cheap and using threads in PHP isn't really a good thing to do.

Answer 2

You can also put your code in another php file and call it using this :

 exec("nohup /usr/bin/php -f your script > /dev/null 2>&1 &");