Laravel Chunk Method Using Eloquent

Question

For processing large database, laravel provides chunk method as here https://laravel.com/docs/5.1/queries#retrieving-results

But how can I use chunk method in this query,

 $data = Inspector::latest('id')
                ->select('id', 'firstname', 'status', 'state', 'phone')
                ->where('firstname', 'LIKE', '%' . $searchtext . '%')
                ->get();

where I'm returning json response like this,

echo json_encode($data);

Any suggestions....

Answer 1

As I understand it the chunk() method is for use when you need to work with a large dataset and take an action on that data chunk by chunk.

From your question, it sounds like you're performing a query then returning the data as JSON so to me, it doesn't sound like you're taking an action on your dataset that requires chunking.

If you want to break up the returned JSON data you should be instead looking at pagination.

You could apply pagination to your query like so:

$data = Inspector::latest('id')
    ->select('id', 'firstname', 'status', 'state', 'phone')
    ->where('firstname', 'LIKE', '%' . $searchtext . '%')
    ->paginate();

You can specify the size of each set by passing a number to the paginate method:

$data = Inspector::latest('id')
    ->select('id', 'firstname', 'status', 'state', 'phone')
    ->where('firstname', 'LIKE', '%' . $searchtext . '%')
    ->paginate(25);

If I've misunderstood and you did actually want to do the chunking, I believe you could do the following:

$data = Inspector::latest('id')
    ->select('id', 'firstname', 'status', 'state', 'phone')
    ->where('firstname', 'LIKE', '%' . $searchtext . '%')
    ->chunk(50, function($inspectors) {
        foreach ($inspectors as $inspector) {
            // apply some action to the chunked results here
        }
    });

Also, if you're returning an eloquent object it will be automatically cast to json so you don't need to perform json_encode() as far as I'm aware.

EDIT

If I've completely misunderstood you and what you actually want to do is this:

{ 1000 records } -> this is the result of your query

split it into this:

{
    { 300 records},
    { 300 records},
    { 300 records},
    { 100 records},
}

Then you want the Collection's chunk method:

$data = Inspector::latest('id')
    ->select('id', 'firstname', 'status', 'state', 'phone')
    ->where('firstname', 'LIKE', '%' . $searchtext . '%')
    ->get() // now we're working with a collection
    ->chunk(300);

Remember you're not working with a Collection until you get the result of the query so if you just call chunk() it will expect a callback, which will be applied to the entire dataset, as you're working with Eloquent.

See for further reading here on the Collection chunk() method here: https://laravel.com/docs/5.3/collections#method-chunk

Otherwise... Could you give some more context to what you're actually doing? It really sounds like you need pagination. What are you doing with the JSON data and how are you invoking the HTTP request, is it via ajax? Why do you need the whole 1000 records all at once?

If you really need the whole dataset of 1000 sent to the client, but 300 at a time then you don't want to use chunk. Think about what chunk() is for in the context of eloquent, it's not for getting chunks to return to the client a chunk at a time until it has the whole data set - it's for applying an action to a chunk then returning the whole set and the point of using it is so it doesn't take up too much memory at one time by loading the whole set to process the action on.

If you want the whole data set bit by bit and pagination won't work for your case (I'm yet to see why!) you will need to invoke the HTTP request several times to get the data bit by bit and specify in each request what you already have and what you need.

Answer 2

Chunk is not for return data by request but for processing large tables on server - here is simple example showing how to use it (tested on Laravel 5.7):

User::chunk(100, function($users) {
    foreach ($users as $user) {
        // processing
    }
});