Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

cURL takes too long to load

Tags:

php

curl

apache

I am calling a REST endpoint in PHP using cURL to fetch some JSON data:

<?php
$ch = curl_init();

curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);

$result = curl_exec($ch);

echo $result;

curl_close($ch);

It takes 2.5 seconds to fetch the data using the above code on my localhost. The same code takes around 7.5 seconds when run on the live server. When the URL is opened directly on a browser it takes only 1.5 seconds.

My question is: Why does it take so long for cURL to fetch data on the live server and how can I solve this problem?

Below is the output of curl_getinfo($ch) on the server:

Array
(
    [content_type] => application/json
    [http_code] => 200
    [header_size] => 420
    [request_size] => 113
    [filetime] => -1
    [ssl_verify_result] => 0
    [redirect_count] => 0
    [total_time] => 7.305496
    [namelookup_time] => 0.150378
    [connect_time] => 0.473187
    [pretransfer_time] => 0.473237
    [size_upload] => 0
    [size_download] => 1291504
    [speed_download] => 176785
    [speed_upload] => 0
    [download_content_length] => -1
    [upload_content_length] => 0
    [starttransfer_time] => 1.787901
    [redirect_time] => 0
    [redirect_url] => 
    [certinfo] => Array
        (
        )

    [primary_port] => 80
    [local_port] => 53962
)
like image 765
littleibex Avatar asked Nov 16 '15 11:11

littleibex


1 Answers

I found the solution to my problem. As I had mentioned in the question, the service was loading the fastest in browsers. So, I checked the 'Request Headers' of the request in the 'Network' tab of Google Chrome Inspector. I copied those headers and used them in my cURL request in PHP. After scraping those headers I found that all I needed to do was to add an Accept-Encoding header. I passed a value of gzip like so:

curl_setopt($ch, CURLOPT_ENCODING, 'gzip');

but setting it to an empty string also works.

curl_setopt($ch, CURLOPT_ENCODING, '');

According to the php.net manual for CURLOPT_ENCODING:

The contents of the "Accept-Encoding: " header. This enables decoding of the response. Supported encodings are "identity", "deflate", and "gzip". If an empty string, "", is set, a header containing all supported encoding types is sent.

like image 127
littleibex Avatar answered Nov 06 '22 09:11

littleibex