MySQL console slow on import of huge SQL files

Question

My import of SQL via the MySQL console is rather slow, and, as our SQL file is increasing every day, I would like to know if there are any alternatives on how to import an SQL file faster.

Changing to Oracle or other systems is not an option, the configuration has to stay the same.

Currently the SQL file is: 1.5 GB. I'm on WAMP with Apache 2.2.14, PHP 5.2.11 and MySQL 5.1.41.

Perhaps the issue is here, import is done by a simple:

mysql -u username -p dbname < sqlfilename.sql

Any suggestions?

Answer 1

Try these http://dev.mysql.com/doc/refman/5.5/en/optimizing-innodb-bulk-data-loading.html

It's maybe an autocommit issue, turn that off then see what happends.

SET autocommit=0 ; source <your dump file> ; COMMIT ;

Answer 2

Having indexes enabled during import will slow your server down to a crawl. ALTER TABLEtablenameDISABLE KEYS; and using ..ENABLE KEYS prior to and after import, will improve import speed, but will take some time to re-create indexes, so it might not be a big speed gain after all.

Also, perhaps using myisam tables (in contrast to innodb with referential integrity options) usually gives better performance, as there is no referential integrity overhead involved.

Personally, I don't use import statement from mysql console, but import sql files using mysql -uUSER -pPASS DBNAME < file.sql, and it works well for me.

Hope it helps.

Answer 3

Switching autocommit off is the first of a series of recommendations given in Bulk Data Loading for InnoDB Tables to speed up restore operations for MySQL.

Instead of switching autocommit off manually at restore time you can already dump your MySQL data in a way that includes all necessary statements right into your SQL file.

The command line parameter for mysqldump is --no-autocommit. You might also consider to add --opt which sets a combination of other parameters to speed up restore operations.

Here is an example for a complete mysqldump command line as I use it, containing --no-autocommit and --opt:

mysqldump -hlocalhost -uMyUser -p'MyPassword' --no-autocommit --opt --default-character-set=utf8 --quote-names  MyDbName  >  dump.sql

For details of these parameters see the reference of mysqldump