Maximum query length to avoid EPIPE error in node.js

Question

I am using felixge/node-mysql module to interact with MySQL database from node.js. I have met the problem when inserting large amounts of data to database in one INSERT request.

Although here there is description of how to add 500,000 records to database using large INSERT-queries, my script fails with the following error with much more small amounts of data:

{ [Error: write EPIPE] code: 'EPIPE', errno: 'EPIPE', syscall:
'write', fatal: true }

I think that this bug is somehow linked with the issue #359 in mysql-node's github Load data infile fails with EPIPE for large files, which is not assigned to someone yet for the last 5 months since creation.

So the what is practical size of queries which are guaranteed to be delivered?

Answer 1

I had this issue just now and it turned out that the Mysql server I was talking to (it was on Amazon RDS) had its max_allowed_packet option set very low. Increasing that solved the issue for me.

I got the clue I needed by running the (very long) query from mysql on the commandline, and searching for the error message I got. I then got instructions for increasing the option on Amazon RDS from here: http://dev.n0ise.net/2012/10/amazon-rds-error-2006-hy000-at-line-xxx-mysql-server-has-gone-away/

See also this SO answer to see what that option is set to on your specific database: https://stackoverflow.com/a/5688506/496046

Answer 2

I've had this problem as well.

I've tried this library: https://github.com/sidorares/node-mysql2

It's a drop in replacement for felixge's mysql driver, and I have not seen the EPIPE errors using it.