Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

cURL loop memory growth

Tags:

linux

php

curl

I have this problem with a loop using cURL where memory grows exponentially. In this example script, it starts using approximately 14MB of memory and ends with 28MB, with my original script and repeating to 1.000.000, memory grows to 800MB, which is bad.

PHP 5.4.5
cURL 7.21.0

for ($n = 1; $n <= 1000; $n++){

    $apiCall = 'https://api.instagram.com/v1/users/' . $n . '?access_token=5600913.47c8437.358fc525ccb94a5cb33c7d1e246ef772';

    $options = Array(CURLOPT_URL => $apiCall,
                     CURLOPT_RETURNTRANSFER => true,
                     CURLOPT_FRESH_CONNECT => true
    );

    $ch = curl_init();
    curl_setopt_array($ch, $options);
    $response = curl_exec($ch);
    curl_close($ch);

    unset($ch);
}
like image 917
user1173536 Avatar asked Aug 15 '12 11:08

user1173536


2 Answers

I think I found a fix to the memory leak. I've got the same problem using curl lib in a PHP script. After repeated calls to curl_exec() function, memory becomes exhausted.

According to a PHP bug report this memory leak may be fixed unsetting the Curl handler after closing it, like next code:

...
curl_close($ch);
unset($ch);
like image 125
Juan Ignacio Pérez Sacristán Avatar answered Sep 22 '22 14:09

Juan Ignacio Pérez Sacristán


This is late, but I recommend against using curl_close in this instance, or if you do, placing it outside the for loop.

We had a similar issue where curl memory started leaking after many loops. We were using curl_multi and closing each of the individual handlers, which caused our memory go bonkers. Overwriting the handler with the curl_init seems to be more than enough. There seems to be an issue with curl_close.

like image 32
clintoncrick Avatar answered Sep 21 '22 14:09

clintoncrick