Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to chunk results from a custom query in Laravel

I have a custom query that grabs data from the old system and maps it to models in the new system. The query looks like this:

$companies = DB::connection('legacy')->select("...");

And since it's a lot of data, I'd like to use Eloquent's chunk feature (just sample code copied from their docs):

User::chunk(200, function($users)
{
    foreach ($users as $user)
    {
        //
    }
});

How do I implement this?


Edit: My code now looks like this, which results in no response:

DB::connection('legacy')->select("SELECT * FROM companies")->chunk(200, function($companies) {
    foreach ($companies as $company) {
        // dd($company);
        $entity       = Entity::firstOrNew(['external_id' => $company->companyKey]);
        $entity->name = $company->companyName;
        $entity->save();
    }
});
like image 838
Donnie Avatar asked Feb 26 '14 16:02

Donnie


People also ask

What is chunk () in laravel?

If your database table has lots of data, chunk() method is the best to use. The chunk() method can be used on the DB facade and also on Eloquent models. The chunk() takes care of fetching a small amount of data at a time and the result is present inside the closure for processing.

What is chunk query?

The chunk() method is part of the query builder that fetches data from the database in smaller numbers/amounts. This is suitable when you have thousands of records your application is working with.

How do you use the chunk method?

Basically, the strategy works by taking a whole bunch of items you have to learn, and chunking them down into a smaller number of items. The simple fact is that you can't remember more than about 5-9 items at once.

How do you select all records from one table that do not exist in another table laravel?

We can get the records in one table that doesn't exist in another table by using NOT IN or NOT EXISTS with the subqueries including the other table in the subqueries.


1 Answers

The chunk feature is only available for Eloquent models and QueryBuilder requests, e.g.

DB::table('tbl')->where('num', '>', 3)->chunk(500, function($rows) {
    // process $rows
});

But it won't work for DB::select('...') request. You need to either use a QueryBuilder request, or use an underlying PDO object to query the database, e.g:

$pdo = DB::getPdo();
$sth = $pdo->prepare("SELECT ....");
$sth->execute();
while ($row = $sth->fetch(PDO::FETCH_ASSOC))
{
    // ...
}
like image 54
AlexM Avatar answered Nov 03 '22 00:11

AlexM