Using Datastore outside of App Engine (python)?

Question

I am used to Google Cloud SQL, where you can connect to a database outside of GAE. Is something like this possible for the GAE datastore, using the Python NDB interface ideally ?

Basically, my use-case is I want to run acceptance tests that pre-populate and clean a datastore.

It looks like the current options are a JSON API or protocol buffers -- in beta. If so, it's kind of a pain then I can't use my NDB models to populate the data, but have to reimplement them for the tests, and worry that they haven't been saved to the datastore in the exact same way as if through the application.

Just checking I'm not missing something....

PS. yes I know about remote_api_shell, I don't want a shell though. I guess piping commands into it is one way, but ugghh ...

Answer 1

Cloud Datastore can be accessed via client libraries outside of App Engine. They run on the "v1 API" which just went GA (August 16, 2016) after a few years in Beta.

The Client Libraries are available for Python, Java, Go, Node.js, Ruby, and there is even .NET.

As a note, GQL language variant supported in DB/NDB is a bit different from what the Cloud Datastore service itself supports via the v1 API. The NDB Client Library does some of its own custom parsing that can split certain queries into multiple ones to send to the service, combining the results client-side.

Take a read of our GQL reference docs.