Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

cURL really slow

Does anybody know why could cURL under php5 be so damn slow to fail even at 45s timeout, downloading a few kb file on a speedO'light server?

The code is here as requested (although I upped the timeouts even more for the script not to fail during execution and changed useragent to Mozilla/4.0 from initial Chrome):

$ch = curl_init('http://www.somesite.com/' . $key);
curl_setopt($ch, CURLOPT_HEADER, false);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_REFERER, 'http://www.somesite.com/somereferer/');
// curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/530.5 (KHTML, like Gecko) Chrome/2.0.172.39 Safari/530.5');
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/4.0');
curl_setopt($ch, CURLOPT_TIMEOUT, 120);
curl_setopt($ch, CURLOPT_DNS_CACHE_TIMEOUT, 600);
like image 250
mangefort Avatar asked Dec 20 '09 21:12

mangefort


2 Answers

hmm, could be a few things, maybe some verbose output will have an error of some kind

curl_setopt($ch, CURLINFO_HEADER_OUT, true);
curl_setopt($ch, CURLOPT_VERBOSE, true); // some output will go to stderr / error_log
curl_setopt($ch, CURLOPT_TIMEOUT, 10);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 5);
$response = curl_exec($ch);
$errStr = curl_error($ch);
$errNum = curl_errno($ch);
$head = curl_getinfo($ch, CURLINFO_HEADER_OUT);
$ci = curl_getinfo($ch);
print_r(array($head, $errStr, $errNum, $ci));

Sometimes the user agent will change how a site responds, may need to do something like:

curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.001 (windows; U; NT4.0; en-US; rv:1.0) Gecko/25250101');
like image 97
jspcal Avatar answered Oct 10 '22 06:10

jspcal


When I set a CONNECTtimeout, I get faster response. Including this option:

curl_setopt($ch, CURLOPT_CONNECTTIMEOUT,1)
like image 4
mistige Avatar answered Oct 10 '22 05:10

mistige