Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

What is the difference between laravel cursor and laravel chunk method?

I would like to know what is the difference between laravel chunk and laravel cursor method. Which method is more suitable to use? What will be the use cases for both of them? I know that you should use cursor to save memory but how it actually works in the backend?

A detailed explanation with example would be useful because I have searched on stackoverflow and other sites but I didn't found much information.

Here is the code snippet's from the laravel documentation.

Chunking Results

Flight::chunk(200, function ($flights) {     foreach ($flights as $flight) {         //     } }); 

Using Cursors

foreach (Flight::where('foo', 'bar')->cursor() as $flight) {     // } 
like image 854
Suraj Avatar asked Aug 02 '17 15:08

Suraj


People also ask

What is the laravel cursor?

The cursor method allows us to iterate through our database using a cursor, which will only execute a single query. While processing large amounts of data, the cursor method may be used to reduce your memory usage greatly. Example foreach (Product::where('name', 'bar')->cursor() as $flight) { //make some stuff }

What is chunk method in laravel?

Basically, Laravel eloquent chunk method break the large group of data set into smaller group of data set (chunks). Suppose, if you work with any big laravel apps and work with large group of records from the database.


2 Answers

We have a comparison: chunk() vs cursor()

  • cursor(): High Speed
  • chunk(): Constant Memory Usage

10,000 records:

+-------------+-----------+------------+ |             | Time(sec) | Memory(MB) | +-------------+-----------+------------+ | get()       |      0.17 |         22 | | chunk(100)  |      0.38 |         10 | | chunk(1000) |      0.17 |         12 | | cursor()    |      0.16 |         14 | +-------------+-----------+------------+ 

100,000 records:

+--------------+------------+------------+ |              | Time(sec)  | Memory(MB) | +--------------+------------+------------+ | get()        |        0.8 |     132    | | chunk(100)   |       19.9 |      10    | | chunk(1000)  |        2.3 |      12    | | chunk(10000) |        1.1 |      34    | | cursor()     |        0.5 |      45    | +--------------+------------+------------+ 
  • TestData: users table of Laravel default migration
  • Homestead 0.5.0
  • PHP 7.0.12
  • MySQL 5.7.16
  • Laravel 5.3.22
like image 196
seyed mohammad asghari Avatar answered Sep 20 '22 04:09

seyed mohammad asghari


Indeed This question might attract some opinionated answer, however the simple answer is here in Laravel Docs

Just for reference:

This is chunk:

If you need to process thousands of Eloquent records, use the chunk command. The chunk method will retrieve a "chunk" of Eloquent models, feeding them to a given Closure for processing. Using the chunk method will conserve memory when working with large result sets:

This is Cursor:

The cursor method allows you to iterate through your database records using a cursor, which will only execute a single query. When processing large amounts of data, the cursor method may be used to greatly reduce your memory usage:

Chunk retrieves the records from the database, and load it into memory while setting a cursor on the last record retrieved so there is no clash.

So the advantage here is if you want to reformat the large record before they are sent out, or you want to perform an operation on an nth number of records per time then this is useful. An example is if you are building a view out/excel sheet, so you can take the record in counts till they are done so that all of them are not loaded into the memory at once and thereby hitting the memory limit.

Cursor uses PHP Generators, you can check the php generators page however here is an interesting caption:

A generator allows you to write code that uses foreach to iterate over a set of data without needing to build an array in memory, which may cause you to exceed a memory limit, or require a considerable amount of processing time to generate. Instead, you can write a generator function, which is the same as a normal function, except that instead of returning once, a generator can yield as many times as it needs to in order to provide the values to be iterated over.

While I cannot guarantee that I understand fully the concept of Cursor, but for Chunk, chunk runs the query at every record size, retrieving it, and passing it into the closure for further works on the records.

Hope this is useful.

like image 20
Oluwatobi Samuel Omisakin Avatar answered Sep 23 '22 04:09

Oluwatobi Samuel Omisakin