Google App Engine and Cloud SQL: Lost connection to MySQL server at 'reading initial communication packet'

Question

I have a Django app on Google App Engine app which is connected to a Google Cloud SQL, using the App Engine authentication.

Most of the time everything works fine, but from time to time the following exception is raised:

OperationalError: (2013, "Lost connection to MySQL server at 'reading initial communication packet', system error: 38")

According to the docs, this error is returned when:

If Google Cloud SQL rejects the connection, for example, because the IP address your client is connecting from is not authorized.

This doesn't make much sense in my case, because the authentication is done by the App Engine server.

What might cause these sporadic errors?

Answer 1

I had a similar issue and ended up contacting Google for help. They explained it happens when they need to restart or move an instance. If the client instance restarted or was moved to another host server (for various versions) the IP’s won’t match and throw that error. They mentioned that the servers may restart for patches, errors and slow downs causing a similar behavior (be it the same error or similar). The server also moves to try and be closer to the instances to increase response times. If you send a request during the move it will throw errors.

They told me I need to code in retry catches incase that happens, similar to how you handle datastore timeouts. Keeping in mind to build in back off mechanics, sending too many request too quickly after a restart could cause a crash.

How often does this happen?

Answer 2

In our case we had renamed the instances incorrectly inside the code. When we changed back to the correct names everything worked fine. Make sure your Cloud SQL instance is named correctly both inside the Google Cloud Console and within the code you use to access it, and make sure that your Cloud SQL instance allows your Google App Engine instance to connect to it it's Access control.