Migrate from Datastore to Firestore native mode

Question

Context, I have a project with datastore which already has information loaded, currently we wanted to use cloud firestore (native mode), but we realized that migration is not possible, what alternatives do I have to use cloud firestore (native mode)?

Answer 1

Update June 16, 2021:

You can now do gcloud datastore export in your first project, followed by gcloud firestore import in your new project. The longer more involved migration below is no longer needed.

Just keep in mind that the Datastore export goes to a Cloud Storage bucket. Make sure that the account running the Firestore import has access to that bucket.

Original answer from 2019

I just migrated from Datastore to Firestore (native mode) for one of my web apps. Here is what I needed to do:

1. Create a new GCP project, as Firestore (native mode) and Datastore can't co-exist in the same project.
2. Migrate the data from Datastore in my old project to Firestore (native mode) in my new project. As of this writing, there are no tools to do that in an automatic way. I wrote Python scripts that read all records from Datastore and wrote them to Firestore in the new project. These scripts ran locally on my machine, using service account keys downloaded from the Cloud Console.

• (Side note: You might be tempted to use gcloud datastore export followed by gcloud firestore import. It seems to work and no error messages pop up when you do. But doc IDs and JSON properties don't translate well. This was a big time-sink for me. Don't go down this road.)

1. Rewrite the data access layer in your app. Firestore (native mode) has a different API than Datastore.

This was a fair amount of work, but it was worth it in my case:

1. I was able to retire a lot of server-side code because the clients can access the database directly.
2. I was able to retire a lot of client-side code for supporting offline mode because the Firestore client library implements it already.

Best of luck!