Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to handle very large arrays in php?

Tags:

php

I have this block of code.

$input = 4;
$list_N = array('0', '1');
for($n=1; $n<=$input; $n++) {

    if($n%2 == 0) {
        $c++;
    }

    $reverse_list_N = array_reverse($list_N);

    $A = array();
    $B = array();

    for($i=0; $i<count($list_N); $i++) {
        $A[] = '0' . $list_N[$i];
        $B[] = '1' . $reverse_list_N[$i];
    }

    $list_N = array_merge($A[], $B[]);

    if($n == 1) {
        $list_N = array('0', '1');
    }
}
$array_sliced = array_slice($list_N, -1*$input, $input);
for($i=0; $i<count($array_sliced); $i++) {
    $output = implode("\n", $array_sliced);
}
echo "<pre>"; print_r($output); echo "</pre>";

What this code does is, it generates following data (starting from (0,1)):

0,1
00, 01, 11, 10
000, 001, 011, 010, 110, 111, 101, 100
....... and so on

When $input = 4; the output is:

1010
1011
1001
1000

And as you can see, after every loop the the elements in the $list_N array doubles than the previous one. And with this pace if $input = 25; then the array would have 33554432 elements which is very huge. And that is the problem I couldn't find a solution of. When $input = 60 I get this error

Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 36 bytes)

on this line

$list_N = array_merge($A, $B);

Even setting the memory limit to 2G did not solve it. So, how do I optimize my code in order to same the memory. Or, is there any other solution ?

Update: Following steps are used to generate the data.

$list_N is an array
$reverse_list is the reverse of $list_N
0 is appended in the beginning of every element in the $list_N array and stored in $A
1 is appended in the beginning of every element in the $reverse_list_N array and stored in $B
Array $A and array $B are merged and is stored in $list_N.
The main loop runs for $input number of times and the last $input number of elements are displayed from the final array.
like image 404
Robbin Lama Avatar asked Apr 21 '17 18:04

Robbin Lama


1 Answers

Solution:

Try using an SplFixedArray!

About:

A SplFixedArrayis

about 37% of a regular "array" of the same size

and

The SplFixedArray class provides the main functionalities of array. The advantage is that it allows a faster array implementation.

From: Documentation Page

Example:

$startMemory = memory_get_usage();
$array = new SplFixedArray(100000);
for ($i = 0; $i < 100000; ++$i) {
    $array[$i] = $i;
}
echo memory_get_usage() - $startMemory, ' bytes';

Further Reading:

Read more: http://nikic.github.io/2011/12/12/How-big-are-PHP-arrays-really-Hint-BIG.html


Messier Solution:

Another solution that could help, which I don't recommend is to override the default memory capacity. You can do this using this:

ini_set('memory_limit', '-1')

Further Reading:

http://php.net/memory_limit

like image 74
Noah Cristino Avatar answered Oct 10 '22 07:10

Noah Cristino