Firestore and Storage Bucket Export/Import between Projects

Question

So in the current Beta Functions of the Google Cloud CLI there is a option to import and export Firestore data. https://firebase.google.com/docs/firestore/manage-data/export-import

Now I made a Export into a Bucket, all worked fine and imported it again, worked well too. Now I have 2 different Projects and wanna import the backup into a different Project, is that possible? Would be good if I only need a billing Account for one Project too.

This line in the Docs says its possible

Once you have export files in Cloud Storage, you can import documents in those files back into your project or to another project.

But the Docs dont cover how exactly its done?

Answer 1

It is possible to import/export between projects. Here are the steps that have worked for me:

First, make sure that the gcloud command line tool is installed. Instructions on set up, and complete information on the export/import process can be read on Firebase's documentation page about Firestore Export and Import.

Before continuing, set the gcloud project to the project from which you want to source your data:

gcloud config set project [PROJECT_ID]

Then, using the Google Cloud Console web application, make sure that a Cloud Storage bucket has been created on the project that will be the source of the data.

For example, for the source bucket, you might create a bucket such as:

gs://my-source-project-export.

You can name the bucket whatever you want, as long as you choose something unique.

Exporting of the source data can then be completed using a command. For example, if you wanted to export just the cameras and radios collections to your my-source-project-export bucket, with a dated directory to identify the export, you include the optional collection-ids flag, as follows:

gcloud beta firestore export gs://my-source-project-export/export-20190113_2109 --collection-ids='cameras','radios'

Omitting the flag would copy ALL the collections.

The gcloud CLI tool should complete the export without issue.

Now, to complete the import, we first switch the gcloud project to the target for our data:

gcloud config set project [PROJECT_ID]

Then, we can attempt the import:

gcloud beta firestore import --collection-ids='cameras','radios' gs://my-source-project-export/export-20190113_2109

The operation may fail due to permission issues. If so, it will report which service account needs to have access to the bucket. To resolve the permission issues, you can simply using the Google Cloud Console Storage Browser to administer the permissions for the source bucket. The required service account must be added to the members list with the role Storage Admin.

Once the permissions are corrected, the operation can be re-attempted. For long running operations, a list of operations and their statuses can be retrieved using the following command:

gcloud beta firestore operations list

Once the import is completed, it may be wise to revoke the permissions granted to the service account, if any, to avoid any unwanted security issues.

Hope that helps.

Answer 2

The accepted answer didn't work for me. No matter what permissions were granted on the source bucket the import always failed with PERMISSION DENIED: The caller does not have permission

The solution was to create another service account. I created a service account on the destination project with Cloud Datastore Import Export Admin and Storage Admin roles. I then added this service account to the source project IAM with the same roles. After doing this the following process worked for me:

gcloud auth activate-service-account --key-file=./mynewserviceaccount.json
gcloud beta firestore export gs://mysourceprojectbucket --project mysourceprojectid
gcloud beta firestore import gs://mysourceprojectbucket/WHATEVER_NAME_FROM_EXPORT --project mydestinationproject