Laravel chunking not reducing PHP memory usage

Question

So I've been trying my hands on laravel's chunking in Eloquent but I've run into a problem. Consider the following code (a much more simplified version of my problem):

$data = DB::connection('mydb')->table('bigdata')
->chunk(200, function($data) {
  echo memory_get_usage();
  foreach($data as $d) {
    Model::create(
      array(
        'foo' => $d->bar,
        ...
        //etc
      ));
  }
}

So when I run the following code my memory outputs look like this:

19039816
21490096
23898816
26267640
28670432
31038840

So without jumping into php.ini and changing the memory_limit value any clue why it isn't working? According to the documentation: "If you need to process a lot (thousands) of Eloquent records, using the chunk command will allow you to do without eating all of your RAM".

I tried unset($data) after the foreach function but it did not help. Any clue as to how I can make use of chunk or did I misinterpret what it does?

Answer 1

Chunking data doesn't reduce memory usage, you need to do it like pagination directly using the database.

Like first fetch starting 200 order by id or something, and after processing first 200, fire that query again with a where clause asking next 200 results.

Answer 2

You can use lazy collections to improve memory uses for a big collection of data. It uses PHP generators under the hood. Take a look at the cursor example here https://laravel.com/docs/5.4/eloquent#chunking-results