Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

LOAD DATA LOCAL INFILE fails - from php, to mysql (on Amazon rds)

We're moving our database from being on the webserver to a separate server (from an Amazon EC2 webserver to an RDS instance.)

We have a LOAD DATA INFILE that worked before that is going to need the LOCAL keyword added now that the database will be on a different machine to the webserver.

Testing on my dev server, it turns out that it doesn't work:

  • I can still LOAD DATA INFILE from php as I have been
  • I can LOAD DATA LOCAL INFILE from mysql commandline (with --local_infile=1)
  • I can't LOAD DATA LOCAL INFILE from php.

Between those 2 things that do work, it rules out:

  • problems with the sql or php code
  • problems with the upload file, including syntax and file permissions
  • mysql server settings problems

The error I get is: ERROR 1148 (42000): The used command is not allowed with this MySQL version (I get that error from the mysql commandline if I don't use --local_infile=1)


A few other bits of relevant info:

  • Ubuntu 12.04, mysql 5.5.24, php 5.3.10
  • I'm using php's mysql_connect (instead of mysqli, because we're planning on using facebook's hiphop compiler which doesn't support mysqli.)
  • Because of that, the connect command needs an extra flag set:

    mysql_connect($dbHost, $dbUser, $dbPass, false, 128);
    
  • I've used phpinfo() to confirm that mysql.allow_local_infile = On
  • I've tried it on Amazon RDS (in case it was a problem in my dev server) and it doesn't work there either. (With the local_infile param turned on.)

The only thing I've read about that I haven't tried is to compile mysql server on my dev server with the flag turned on to allow local infile... but even if I get that working on my dev server it's not going to help me with Amazon RDS. (Besides which, LOAD DATA LOCAL INFILE does work from the mysql commandline.)


It seems like it's specifically a problem with php's mysql_connect()

Anybody using LOAD DATA LOCAL INFILE (maybe from Amazon RDS) that knows the trick to getting this to work?

like image 623
Redzarf Avatar asked Oct 22 '12 17:10

Redzarf


People also ask

How do I import data from MySQL to Amazon RDS?

Import data from an external MySQL DB instance into an Amazon RDS DB instance by first dumping it using the mysqldump command line utility. You then import it using the LOAD DATA LOCAL INFILE command and use replication to bring the instances into sync.

Does replication work between MySQL and Amazon RDS?

For more information, see Replication compatibility between MySQL versions in the MySQL documentation. The first step in the process of migrating a large amount of data to an Amazon RDS MySQL or MariaDB DB instance with minimal downtime is to create a copy of the source data.

How can I verify that mysqldump data has been successfully imported?

If you used any data-formatting options with mysqldump when you initially dumped the table, you must use the same options with mysqlimport or LOAD DATA LOCAL INFILE to ensure proper interpretation of the data file contents. Run a simple SELECT query against one or two of the tables in the imported database to verify that the import was successful.

Why is local infile disabled by default in MySQL?

LOAD DATA LOCAL INFILE is disabled by default because it poses a security risk. A malicious server or proxy could send a fake “local infile request” packet to the client and read any file that the client has permission to open. For more information, see the MySQL documentation.


1 Answers

I've given up on this, as I think it's a bug in php - in particular the mysql_connect code, which is now deprecated. It could probably be solved by compiling php yourself with changes to the source using steps similar to those mentioned in the bug report that @eggyal mentioned: https://bugs.php.net/bug.php?id=54158

Instead, I'm going to work around it by doing a system() call and using the mysql command line:

$sql = "LOAD DATA LOCAL INFILE '$csvPathAndFile' INTO TABLE $tableName FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '\\\"' ESCAPED BY '\\\\\\\\' LINES TERMINATED BY '\\\\r\\\\n';";
system("mysql -u $dbUser -h $dbHost --password=$dbPass --local_infile=1 -e \"$sql\" $dbName");

That's working for me.

like image 190
Redzarf Avatar answered Oct 08 '22 19:10

Redzarf