Decrease memory usage of recursive functions in PHP

Question

I have a recursive function in PHP where it loops in an API where it just allows you to recover 200 records of it at a time.

But since this API has a very high response latency we decided to use a local intermediate database where I add these records and the same will be shown on the site.

However, as this API has more than 30000 records the recursive function it consumes a lot of memory due to it in the case of 30000 records it would have to be recursively invoked more than 1500 times and it ends up giving the famous StackOverflow.

I wonder if there is a manual way to clear the memory of this function by calling it again without losing the value of it.

code example:

public function recursive ($index = 0, $offset = 200) {
   $api = GetConectApi ($index, offset)-> Getrecords ();
   foreach ($api $value) {
      \\Here is my necessary loop
   }
   if (count ($API) > 0) {
      $this Recursive ($index + 200, $offset + 200);
   }
}

I would like to find a way that when it called the recursive function again eliminated the previous allocation without losing the reference value that was passed.

Answer 1

To expand on user3720435's answer, you are using a lot of memory by creating a new $api variable each time you run the function. To understand why, let's "unroll" the code - imagine it was all written out in sequence with no function calls:

$api1 = GetConectApi ($index1, offset1)-> Getrecords ();
foreach ($api1 => $value1) {
    // Here is my necessary loop
}
if (count ($api1) > 0) {
    // RECURSION HAPPENS HERE
    $index2 = $index1 + 200, $offset2 = $offset1 + 200
    $api2 = GetConectApi ($index, offset)-> Getrecords ();
    foreach ($api2 => $value2) {
        // Here is my necessary loop
    }
    if (count ($api2) > 0) {
        // RECURSE AGAIN, AND AGAIN, AND AGAIN
    }
}

Note that I've renamed all the variables as $api1, $api2, etc. That's because each time you run the function, $api is actually a different variable. It has the same name in your source code, but it doesn't represent the same piece of memory.

Now, PHP doesn't know you're not going to use $api1 again when you create $api2, so it has to keep both in memory; as you end up with more and more sets of data, it needs more and more memory.

user3720435's suggestion is to add unset($api) before the recursion:

$api = GetConectApi ($index, offset)-> Getrecords ();
foreach ($api => $value) {
      // Here is my necessary loop
}
if (count ($api) > 0) {
      unset($api);
      // code as before
}

This tells PHP that you don't need that memory any more, so as it recurses, it won't build up. You'll still build up multiple copies of $index and $offset, but these are likely to be very small in comparison.

All that being said, it's not clear why you need recursion here at all. The whole thing could actually be changed to a simple loop:

do {
    $api = GetConectApi ($index, offset)-> Getrecords ();
    foreach ($api => $value1) {
       // Here is my necessary loop
    }
    $index = $index + $offset;
} while (count ($api) > 0)

A do..while loop always executes once, and then keeps repeating until the condition becomes false. Unrolled it looks like this:

// do...
    $api = GetConectApi ($index, offset)-> Getrecords ();
    foreach ($api => $value1) {
       // Here is my necessary loop
    }
    $index = $index + $offset;
if (count ($api) > 0) { // while...
$api = GetConectApi ($index, offset)-> Getrecords ();
    foreach ($api => $value1) {
       // Here is my necessary loop
    }
    $index = $index + $offset;
}
if (count ($api) > 0) { // while...
// etc

Note that we don't need to allocate any extra memory as we go round the loop, because we haven't entered a new function - we're just using the same variable over and over again.