PHP only store 1048576 characters in array

Question

I'm working on a project that parses huge text files and store some of the information in MySQL DB. I noticed one of the field was missing info when it was displayed, however, when checking the DB (from phpmyadmin), it shows the data is complete, so the problem must be with the php, before the length of the array field is 1048576, whereas there are 1235597 characters in DB.

My php info shows memory_limit is 2048M, Mysql config shows: [mysqld] key_buffer = 32M max_allowed_packet = 32M

I don't know what else could have caused the problem... someone please help!!!

Thanks, Sean

Answer 1

PDO's default buffer size is 1 MB (1048576), try bumping it up to 2 MB (2097152)

If you are using PDO directly, pass this as the 4th argument

$pdo = new PDO(
    $dsn,
    $username,
    $password,
    array(PDO::MYSQL_ATTR_MAX_BUFFER_SIZE => 2097152)
);

If you are using Laravel, this can be done via the config/database.php file by adding an array of options to your connection

// ...
    'mysql' => array(
        'driver'    => 'mysql',
        // ...
        'charset'   => 'utf8',
        'collation' => 'utf8_unicode_ci',
        'prefix'    => '',
        'options'   => array(
            PDO::MYSQL_ATTR_MAX_BUFFER_SIZE => 2097152
        ),
    ),
// ...