Why does manage.py syncdb fail to connect to google cloud sql database?

Question

During a recent update of an app on Google App Engine I was in the process of updating the database with:

SETTINGS_MODE='prod' ./manage.py syncdb

This worked last time I ran it, but now I am getting the following error:

Traceback (most recent call last):
  File "./manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/Library/Python/2.7/site-packages/django/core/management/__init__.py", line 399, in execute_from_command_line
    utility.execute()
  File "/Library/Python/2.7/site-packages/django/core/management/__init__.py", line 392, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/Library/Python/2.7/site-packages/django/core/management/base.py", line 242, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/Library/Python/2.7/site-packages/django/core/management/base.py", line 285, in execute
    output = self.handle(*args, **options)
  File "/Library/Python/2.7/site-packages/django/core/management/base.py", line 415, in handle
    return self.handle_noargs(**options)
  File "/c/clients/green_rentals/code/green-rental/south/management/commands/syncdb.py", line 92, in handle_noargs
    syncdb.Command().execute(**options)
  File "/Library/Python/2.7/site-packages/django/core/management/base.py", line 285, in execute
    output = self.handle(*args, **options)
  File "/Library/Python/2.7/site-packages/django/core/management/base.py", line 415, in handle
    return self.handle_noargs(**options)
  File "/Library/Python/2.7/site-packages/django/core/management/commands/syncdb.py", line 57, in handle_noargs
    cursor = connection.cursor()
  File "/Library/Python/2.7/site-packages/django/db/backends/__init__.py", line 159, in cursor
    cursor = util.CursorWrapper(self._cursor(), self)
  File "/c/downloads/python/google_appengine/google/storage/speckle/python/django/backend/base.py", line 263, in _cursor
    if not self._valid_connection():
  File "/c/downloads/python/google_appengine/google/storage/speckle/python/django/backend/base.py", line 258, in _valid_connection
    return super(DatabaseWrapper, self)._valid_connection()
AttributeError: 'super' object has no attribute '_valid_connection'

I tried updating the Google App Engine SDK to the latest (1.8.8) and I also tried re-authenticating with oauth (that worked). Still getting the same message though. Any clues as to why?

Thanks in advance!

Edit:

It looks like the settings I was using for the database connection have changed recently. I used to use:

DATABASES = {
    'default': {
        'ENGINE': 'google.appengine.ext.django.backends.rdbms',
        'INSTANCE': 'project-id:instance-id',
        'NAME': 'name',
    }
}

But now the recommendation is:

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.mysql',
        'HOST': '/cloudsql/your-project-id:your-instance-name',
        'NAME': 'database-name',
        'USER': 'mysql-user',
    }
}

After updating to that, I now get a new error:

Traceback (most recent call last):
  File "./manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/Library/Python/2.7/site-packages/django/core/management/__init__.py", line 399, in execute_from_command_line
    utility.execute()
  File "/Library/Python/2.7/site-packages/django/core/management/__init__.py", line 392, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/Library/Python/2.7/site-packages/django/core/management/base.py", line 242, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/Library/Python/2.7/site-packages/django/core/management/base.py", line 285, in execute
    output = self.handle(*args, **options)
  File "/Library/Python/2.7/site-packages/django/core/management/base.py", line 415, in handle
    return self.handle_noargs(**options)
  File "/c/clients/green_rentals/code/green-rental/south/management/commands/syncdb.py", line 89, in handle_noargs
    db.connection_init() 
  File "/c/clients/green_rentals/code/green-rental/south/db/mysql.py", line 180, in connection_init
    cursor = self._get_connection().cursor()
  File "/Library/Python/2.7/site-packages/django/db/backends/__init__.py", line 159, in cursor
    cursor = util.CursorWrapper(self._cursor(), self)
  File "/Library/Python/2.7/site-packages/django/db/backends/__init__.py", line 129, in _cursor
    self.ensure_connection()
  File "/Library/Python/2.7/site-packages/django/db/backends/__init__.py", line 124, in ensure_connection
    self.connect()
  File "/Library/Python/2.7/site-packages/django/db/utils.py", line 99, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/Library/Python/2.7/site-packages/django/db/backends/__init__.py", line 124, in ensure_connection
    self.connect()
  File "/Library/Python/2.7/site-packages/django/db/backends/__init__.py", line 112, in connect
    self.connection = self.get_new_connection(conn_params)
  File "/Library/Python/2.7/site-packages/django/db/backends/mysql/base.py", line 435, in get_new_connection
    conn = Database.connect(**conn_params)
  File "/Library/Python/2.7/site-packages/MySQLdb/__init__.py", line 81, in Connect
    return Connection(*args, **kwargs)
  File "/Library/Python/2.7/site-packages/MySQLdb/connections.py", line 187, in __init__
    super(Connection, self).__init__(*args, **kwargs2)
django.db.utils.OperationalError: (2002, "Can't connect to local MySQL server through socket '/cloudsql/your-project-id:your-instance-name' (2)")

(Where your-project-id:your-instance-name have been changed appropriately).

Answer 1

Intro

I'm going to leave a more detailed answer here for those that have also been tripped up by this, I realise the other one has already been accepted but hopefully this will help someone. We are on Django 1.6 at the time of writing.

It basically comes down to this: the Django <-> CloudSQL docs for Google App Engine are currently out of date. They utilise the OAuth2 connection mechanism which is clunky at best and a complete pain if you're using the syncdb management command from inside a headless Vagrant/virtual machine as it requires you to have a Javascript enabled browser to actually retrieve the token (unless you play about with the gflags library's global state, but that's a story for another day) and this is obviously quite a bother on the command line (I believe only elinks has CLI JS support and the Ubuntu repo versions of that don't include the js support..)

Setup within the Google App Engine Web Console

If you go to this link you will find instructions on the new way to connect to instances. This works by doing some clicking about in the web console:

1. Assigning a public IP to the Cloud SQL instance (think of these in terms of AWS Elastic IPs if that helps).
2. Allowing your local IP address/range access to the Cloud SQL instance.

Correct your Django settings

For production instances running on App Engine itself:

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.mysql',
        'HOST': '/cloudsql/project-name:instance-name',
        'NAME': 'name_of_pre_created_database',
        'USER': 'root_or_pre_created_user',
    }
}

For development instances that connect to a production Cloud SQL instance:

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.mysql',
        'HOST': '198.198.88.88', 
        'NAME': 'name_of_pre_created_database',
        'USER': 'root_or_pre_created_user',
        'PASSWORD': 'password_of_pre_created_or_root_user',
    }
}

Where 198.198.88.88 is the IP address that Google automatically assign to your Cloud SQL instance after you request it (see above). Remember that you'll need to connect from an IP that you have allowed to access that specific Cloud SQL instance.

And with this, google.appengine.ext.django.backends.rdbms can bite the dust (breathe sigh of relief).