Faster App Engine Development Datastore Alternative

Question

Is there a way to use a real database(SQLite, Mysql, or even some non-relational one) as datastore for development, instead of memory/file datastore that is provided.

I saw few projects, GAE-SQLite(did not seem to be working) and one tip about accessing production datastore using remote api (still pretty slow for large datasets).

Answer 1

MongoDB works great for that. You will need:

• The MongoDB stub: http://github.com/mongodb/mongo-appengine-connector
• MongoDB: http://www.mongodb.org/display/DOCS/Downloads
• Some code to set it up like:

code:

import datastore_mongo_stub

os.environ['APPLICATION_ID'] = 'test'

datastore = datastore_mongo_stub.DatastoreMongoStub(
    os.environ['APPLICATION_ID'], 'woot', '', require_indexes=False)

apiproxy_stub_map.apiproxy.RegisterStub('datastore_v3', datastore)

But if you're looking for truly faster development (like I was) the datastore is actually not the issue as much is the single threaded web server. I tried to replace it with spawning but that was a little too hard. You could also try to set up TyphoonAE which will mimic the appengine stack with open alternatives.

Be aware that if you do any of these, you might lose some of the exact behavior the current tools provide, meaning that if you deploy you could get results you didn't expect. In other words; make sure you know what you're doing :-)

Answer 2

The Google App Engine SDK for Python now bundles support for SQLite. See the official docs for more information.

Answer 3

bdbdatastore is an alternative datastore backend that's considerably better than the one built in to the development server, although the datastore is far from being the only problem with the dev server when it comes to handling large applications.