Doctrine2 $em->persist($entity) on foreach loop

Question

I'm currently in a spot, where I need to create or update entities in a foreach loop.

So I'm doing the following (short code):

foreach ($dataset as $data) {
    $entity = new Entity();

    // ---- Some setting operations on the entity

    $em->persist($entity);
}

$em->flush();

The thing I was expecting is that Doctrine manages the entities and then with one statement inserts the entities into the table.

But it occurs, that Doctrine makes one statement for each created entity. Since the $dataset array can be pretty big (a lot of entities created), I would like to have it packed into one statement.

How can I achieve this?

Answer 1

From the Doctrine documentation, it says that inserts are best performed with batch. And its a development of @AlterPHP 's answer.

You could use :

$batchSize = 20;

for ($i = 1; $i <= 10000; ++$i) {

    $car = new Car();
    // ... set number of wheels, but should always be to 4 right ?

    $em->persist($car);

    if (($i % $batchSize) === 0) {
        $em->flush();
        $em->clear(Car::class); // Detaches all Car objects from Doctrine!
    }
}

$em->flush(); // Persist objects that did not make up an entire batch
$em->clear(Car::class);

PS: i just read that from Doctrine 13.1. Bulk Inserts section. Now all you'll need is a bigger parking !

Answer 2

As suggested by greg0ire, this link describes how Doctrine optimizes INSERT statements : https://www.slideshare.net/jwage/doctrine-2-not-the-same-old-php-orm/47-sflive2010_Insert_Performance_Inserting_20 (have a look from slide #47). It uses transactions but doesn't group INSERT of same objects in a unique statement.

If you really need to divide the amount of data you pass to your DB server at once, I suggest you process EntityManager::flush() every x statement.

Answer 3

Change this code:

foreach ($dataset as $data) {
    $entity = new Entity();
    // ---- Some setting operations on the entity
    $em->persist($entity);
}

to:

foreach ($dataset as $data) {
    $entity = new Entity();
    // ---- Some setting operations on the entity
    $em->persist($entity);
    $em->flush();
    $em->clear();
}