Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Unserialize With Memory/Size Constraints

Is there a way to use unserialize with a memory/size limit?

Currently we have:

$data = unserialize($_SESSION['visits']);

and we occasionally get:

PHP Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 17645568 bytes) in Unknown on line 0

when a visitor has had a lot of visits in a short period of time (session value stores information about each page visited).

If the length of $_SESSION['visits'] is above a million characters it causes the issue so I can do a simple check on that but is there a better solution than this:

 if(strlen($_SESSION['visits']) <= 1000000) {
    $data = unserialize($_SESSION['visits']);
} else {
    $data = array();
}

I thought try catch might behave better but it didn't get caught:

try{
    $data = unserialize($_SESSION['vists']);
} catch(\Exception $exception){
    error_log('Caught memory limit');
}

The answer to this question is not to increase the memory size.

like image 533
user3783243 Avatar asked Jun 16 '20 14:06

user3783243


2 Answers

There are two options:

a) unserialize in another phpprocess

Which can fail with memory limit error and possibly return back only the data you are interested in.

How?

  1. create custom php script/file that:
    1. call autoloader/init app
    2. deserialize the passed value thru file or $argv
  2. call that script using exec()

to address the question requirement, you can:

  • either return the memory usage from the custom script (and check if that memory will be available in the parent script)
  • OR return only the data you are interested in back thru file or stdout

b) use custom parser

Like https://github.com/xKerman/restricted-unserialize , which allows:

  • deserialize by individual values and check if you have enough memory before (same problem, but with small data you can check much better)
  • traverse the data without unserializing them completely (thus not using extra memory)

c) use database

The two options above are solution to your requirement. However my strong advise is to store the session/visits data in a database and then store only an unique id to them.

like image 81
mvorisek Avatar answered Nov 08 '22 12:11

mvorisek


The most memory efficient you'll get is probably by storing everything in a string, packed in binary, and use manual indexing to it.And for this you can make use of pack() method.

Memory Usage Differences

$a = memory_get_usage();
$data = serialize(array(1 => 1, 0 => 2, 3 => 3));
$b = memory_get_usage();
$c = $b - $a;
echo $c; //Outputs 296

And when same data packed in the form of binary string.

$a = memory_get_usage();
$data = pack("C*",array(1 => 1, 0 => 2, 3 => 3));
$b = memory_get_usage();
$c = $b - $a;
echo $c; //Outputs 72

Saves your memory a lot more than expected and is very efficient.

like image 39
Kunal Raut Avatar answered Nov 08 '22 14:11

Kunal Raut