Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

PHP Multiple Curl Requests

I'm currently using Curl for PHP a lot. It takes a lot of time to get results of about 100 pages each time. For every request i'm using code like this

$ch = curl_init();  // get source  curl_close($ch); 

What are my options to speed things up?

How should I use the multi_init() etc?

like image 834
Simon Avatar asked Oct 10 '10 11:10

Simon


People also ask

How do I send multiple cURL requests?

The solution to this is to use the xargs command as shown alongside the curl command. The -P flag denotes the number of requests in parallel. The section <(printf '%s\n' {1.. 10}) prints out the numbers 1 – 10 and causes the curl command to run 10 times with 5 requests running in parallel.

How do you use cURL multi?

To use the multi interface, you must first create a 'multi handle' with curl_multi_init. This handle is then used as input to all further curl_multi_* functions. With a multi handle and the multi interface you can do several simultaneous transfers in parallel. Each single transfer is built up around an easy handle.

Is PHP cURL asynchronous?

Short answer is no it isn't asynchronous.


2 Answers

  • Reuse the same cURL handler ($ch) without running curl_close. This will speed it up just a little bit.
  • Use curl_multi_init to run the processes in parallel. This can have a tremendous effect.
like image 163
Emil Vikström Avatar answered Sep 21 '22 19:09

Emil Vikström


take curl_multi - it is far better. Save the handshakes - they are not needed every time!

like image 40
urs_baer Avatar answered Sep 23 '22 19:09

urs_baer