Failed to get existing workspaces: querying Cloud Storage failed: storage: bucket doesn't exist

Question

I was using terraform in cloud build, but it fails at this step

steps:
  # Terraform
  - id: 'configure_terraform'
    name: node:10.16.3
    entrypoint: "node"
    args: ["./create_terraform_config.js",
           "../terraform/override.tf",
           "${_TERRAFORM_BUCKET_NAME}",
           "${_TERRAFORM_BUCKET_PATH}"]
    dir: "app/scripts"
  - id: 'init_terraform'
    name: hashicorp/terraform:light
    args: ["init"]
    dir: "app/terraform"

Initializing the backend...

Successfully configured the backend "gcs"! Terraform will automatically
use this backend unless the backend configuration changes.

Error: Failed to get existing workspaces: querying Cloud Storage failed: storage: bucket doesn't exist

Answer 1

This will fix the issue

terraform init -reconfigure

reference: https://github.com/hashicorp/terraform/issues/23532#issuecomment-560493391

Answer 2

This is what worked for me:

gcloud auth application-default login --project $PROJECT

Normally I omit the --project arg but that still generated the error. I thought it was only used for billing/quota. This may be a bug related to my specific tool versions:

Google Cloud SDK 387.0.0
Terraform v1.1.2

Answer 3

1. State bucket must be pre-existing. GCS backend bucket must pre-exist

If it doesn't exist, create the state bucket and version it, using the following commands:

gsutil mb -p <projectId> -c <storage-class> -l <region> -b gs://<bucket-name>
gsutil versioning set on gs://<bucket-name>

2. As specified in the above answer execute terraform init with -reconfigure option.