Allowed memory size exhausted in PHP for loop

Question

I'm facing a fatal error while I'm trying to manipulate a huge array of arrays in PHP and return the result as a response of an HTTP POST request:

Allowed memory size of 536870912 bytes exhausted

I have already tried to set ini_set('memory_limit', '-1'); in order to see if I get the result, but I didn't get any type of response. Postman crashed all the times that I tried to make the POST request.

The starting structure of the array is this one. The body size is around 25mb. The main array contains around 22k arrays with this structure, I have just included 2:

Array
(
    [0] => Array
        (
            [id] => 14
            [isActive] => 1
            [personId] => 0023fff16d353d16a052a267811af53bc8bd42f51f2266a2904ca41db19dfd32_0
            [gender] => m
            [age] => 69
            [linedata_0] => 2018-03-01 17:15:18, 155.59, 294.076; 2018-03-01 17:16:04, 502.968, 249.947; 2018-03-01 17:16:44, 276.837, 270.593; 2018-03-01 17:17:28, 431.68, 371.14; 2018-03-01 17:17:34, 851.622, 355.915
        )

    [1] => Array
        (
            [id] => 180
            [isActive] => 1
            [personId] => 02659982ae8286409cc5bb283089871b62f2bafbbad517941d64e77ecf2b62b1_0
            [gender] => m
            [age] => 69
            [linedata_0] => 2018-03-01 13:20:05, 155.599, 293.841; 2018-03-01 13:20:48, 495.468, 249.582; 2018-03-01 13:21:28, 258.791, 260.748; 2018-03-01 13:23:20, 859.061, 352.237; 2018-03-01 13:23:32, 56.1404, 269.858
        )
)

Here below the php part for manipulate the array in order to have the expected final result by explode the timestamp and coordinates for each user:

$final_result = [];

foreach($query_result as $row)
{
    $line_datas =explode(";",$row["linedata_0"]);
    $linedata = [];
    $final = [];
    $d = [];

    for($s =0; $s < count($line_datas); $s++){
        $line_data = explode(",",$line_datas[$s]);
        $d["timestamp"] = utf8_encode($line_data[0]);
        $d["x"]= utf8_encode($line_data[1]);
        $d["y"] = utf8_encode($line_data[2]);

        array_push($linedata,$d);
    }

    $final["id"]= $row["id"];
    $final["isActive"]= $row["isActive"];
    $final["personId"]= utf8_encode($row["personId"]);
    $final["name"] = NULL;
    $final["gender"] = utf8_encode($row["gender"]);
    $final["age"] = utf8_encode($row["age"]);
    $final["linedata"]=$linedata;

    array_push($final_result, $final);
}

return $final_result;

As it seems to me there are no infinite loop or bad practices that can justify a memory issue. The only real problem could be the size of the array that need to be manipulated.

Any suggestions?

Answer 1

You are collecting a large amount of data into the array, and only then returning it.

If you instead, collect a single '$final' item, and yield it inside the foreach-loop, rather than putting it into an ever-increasing sized variable, you will still be able to foreach around the function call.

Here is a simplistic example, where $i stands in as a sample returning value instead of your '$final' array of collected data.

<?php
function count_one_to_three() {
    for ($i = 1; $i <= 3; $i++) {
        // Note that $i is preserved between yields.
        yield $i;
    }
}

$generator = count_one_to_three();
foreach ($generator as $value) {  // you can also foreach(count_one_to_three() as $value)
    echo "$value\n";
}

Information on 'yield' in PHP