Is making asynchronous HTTP requests possible with PHP?

Question

I have a PHP script that needs to download several files from a remote server. At the moment I just have a loop downloading and processing the files with cURL, which means that it doesn't start downloading one file until the previous one is finished - this increases the script run time significantly.

Would it be possible to start several instances of cURL, for example, to asynchronously download these files at the same time without waiting for the previous one to finish? If so, how would this be accomplished?

Answer 1

Check out curl-easy. It supports both blocking and not-blocking requests in parallel or single request at once. Also, it is unit-tested, unlike many simple or buggy libraries.

Disclosure: I am the author of this library. The library has it's own test suite so I'm pretty confident it is robust.

Also, check out example of use below:

<?php // We will download info about 2 YouTube videos: // http://youtu.be/XmSdTa9kaiQ and // http://youtu.be/6dC-sm5SWiU  // Init queue of requests $queue = new cURL\RequestsQueue; // Set default options for all requests in queue $queue->getDefaultOptions()     ->set(CURLOPT_TIMEOUT, 5)     ->set(CURLOPT_RETURNTRANSFER, true); // Set callback function to be executed when request will be completed $queue->addListener('complete', function (cURL\Event $event) {     $response = $event->response;     $json = $response->getContent(); // Returns content of response     $feed = json_decode($json, true);     echo $feed['entry']['title']['$t'] . "\n"; });  $request = new cURL\Request('http://gdata.youtube.com/feeds/api/videos/XmSdTa9kaiQ?v=2&alt=json'); $queue->attach($request);  $request = new cURL\Request('http://gdata.youtube.com/feeds/api/videos/6dC-sm5SWiU?v=2&alt=json'); $queue->attach($request);  // Execute queue $queue->send(); 

Answer 2

Yes.

There is the multirequest PHP library (or see: archived Google Code project). It's a multi-threaded CURL library.

As another solution, you could write a script that does that in a language that supports threading, like Ruby or Python. Then, just call the script with PHP. Seems rather simple.