PHP 7 - execute code while waiting for curl

Question

The following is my working code sample. Just add your own sleep.php which will sleep($_GET['sleep']);

class MultiCurl {
    private $mc;
    private $running;
    private $execStatus;

    public function __construct() {
        $this->mc = curl_multi_init();
    }

    public function addCurl($ch) {
        $code = curl_multi_add_handle($this->mc, $ch);

        if ($code === CURLM_OK || $code === CURLM_CALL_MULTI_PERFORM) {
            do {
                $this->execStatus = curl_multi_exec($this->mc, $this->running);
            } while ($this->execStatus === CURLM_CALL_MULTI_PERFORM);

            return $this->getKey($ch);
        }
        return null;
    }

    public function getNextResult() {
        if ($this->running) {
            while ($this->running && ($this->execStatus == CURLM_OK || $this->execStatus == CURLM_CALL_MULTI_PERFORM)) {
                usleep(2500);
                curl_multi_exec($this->mc, $this->running);

                $responses = $this->readResponses();
                if ($responses !== null) {
                    return $responses;
                }
            }
        } else {
            return $this->readResponses();
        }

        return null;
    }

    private function readResponses() {
        $responses = [];
        while ($done = curl_multi_info_read($this->mc)) {
            $key = $this->getKey($done['handle']);

            $done['response'] = curl_multi_getcontent($done['handle']);
            $done['info'] = curl_getinfo($done['handle']);
            $error = curl_error($done['handle']);
            if ($error) {
                $done['error'] = $error;
            }

            $responses[$key] = $done;

            curl_multi_remove_handle($this->mc, $done['handle']);
            curl_close($done['handle']);
        }

        if (!empty($responses)) {
            return $responses;
        }

        return null;
    }

    private function getKey($ch) {
        return (string)$ch;
    }
}

function getHandle($url) {
    $ch = curl_init();
    curl_setopt_array($ch, [
        CURLOPT_URL => $url,
        CURLOPT_RETURNTRANSFER => true,
        CURLOPT_CONNECTTIMEOUT => 5
    ]);
    return $ch;
}

$totalTime = microtime(true);

$multi = new MultiCurl();

$keys = [];
$addCurlHandles = microtime(true);
$keys[] = $multi->addCurl(getHandle('http://localhost/sleep.php?sleep=5'));
for ($i = 0; $i < 5; $i++) {
    $keys[] = $multi->addCurl(getHandle('http://localhost/sleep.php?sleep=' . random_int(1, 4)));
}
echo 'Add curl handles: ' . (microtime(true) - $addCurlHandles) . "\n";

/**/
$loop = microtime(true);
while (microtime(true) - $loop < 2) {
    usleep(100);
}
echo 'Loop: ' . (microtime(true) - $loop) . "\n";
/**/

$getResults = microtime(true);
while ($result = $multi->getNextResult()) {
    foreach ($result as $key => $response) {
        echo $response['response'] . "\n";
    }
}
echo 'Get results: ' . (microtime(true) - $getResults) . "\n";

echo 'Total time: ' . (microtime(true) - $totalTime) . "\n";

Now play around with the for loop calling $multi->addCurl. When I add 4 handles, the output is something like

Add curl handles: 0.0007021427154541
Loop: 2.0000491142273
Slept 1
Slept 3
Slept 3
Slept 4
Slept 5
Get results: 5.0043671131134
Total time: 7.0052678585052

But when I add 5 or more, the output is

Add curl handles: 0.0014941692352295
Loop: 2.00008893013
Slept 1
Slept 2
Slept 4
Slept 4
Slept 4
Slept 5
Get results: 3.0007629394531
Total time: 5.0025300979614

As you can see, the later does more work but finishes faster because the 5 second sleep request was actually sent before the 2 second while loop started working.

With the smaller number of handles, calling curl_multi_select and curl_multi_exec in a loop until curl_multi_select doesn't return -1 has resolved this but it's very unreliable. It doesn't work at all on another computer and will sometimes get stuck with curl_multi_select always returning -1.

Answer 1

I have found a hack solution. It is to check pretransfer_time using curl_getinfo.

I have published the source code on github: https://github.com/rinu/multi-curl

Better, cleaner solutions are still very welcome.