Migrating from remote-state to backend in Terraform 0.9

Question

I downloaded terraform 0.9 and tried to follow the migration guide to move from remote-state to backend

But it doesn't seem to work. I replaced:

data "terraform_remote_state" "state" {
  backend = "s3"
  config {
    bucket = "terraform-state-${var.environment}"
    key = "network/terraform.tfstate"
    region = "${var.aws_region}"
  }
}

with

terraform {
  backend "s3" {
    bucket = "terraform-backend"
    key = "network/terraform.tfstate"
    region = "us-west-2"
  }
}

yet when I run terraform init in one of my environment folders, I get:

Deprecation warning: This environment is configured to use legacy remote state. Remote state changed significantly in Terraform 0.9. Please update your remote state configuration to use the new 'backend' settings. For now, Terraform will continue to use your existing settings. Legacy remote state support will be removed in Terraform 0.11.

You can find a guide for upgrading here:

https://www.terraform.io/docs/backends/legacy-0-8.html

I also had to drop the variable interpolation since this is not allowed anymore. Does that mean that one S3 Bucket is used for multiple environments? What have I missed here?

Answer 1

Per upgrade guide (https://www.terraform.io/docs/backends/legacy-0-8.html) after terraform init you also have to run terraform plan to finalize the migration, which will update the remote state file at s3.

As for configuring for multiple environments we ended up using a wrapper shell script with passing in parameters for ${application_name}/${env}/${project}, and using partial configuration.

For a project structure like this:

├── projects
│   └── application-name
│       ├── dev
│       │   ├── bastion
│       │   ├── db
│       │   ├── vpc
│       │   └── web-cluster
│       ├── prod
│       │   ├── bastion
│       │   ├── db
│       │   ├── vpc
│       │   └── web-cluster
│       └── backend.config
└── run-tf.sh

for each application_name/env/component = folder (i.e. dev/vpc) we added a placeholder backend configuration file like this: backend.tf:

terraform {
    backend "s3" {
    }
}

Where the folder content for each component will look like this:

│       ├── prod
│       │   ├── vpc
│       │   │   ├── backend.tf
│       │   │   ├── main.tf
│       │   │   ├── outputs.tf
│       │   │   └── variables.tf

At "application_name/" or "application_name/env" level we added a backend.config file, like this:

bucket     = "BUCKET_NAME"
region     = "region_name"
lock       = true
lock_table = "lock_table_name"
encrypt    = true

Our wrapper shell script expects parameters application-name, environment, component, and the actual terraform cmd to run.

The content of run-tf.sh script (simplified):

#!/bin/bash

application=$1
envir=$2
component=$3
cmd=$4

tf_backend_config="root_path/$application/$envir/$component/backend.config"

terraform init -backend=true -backend-config="$tf_backend_config" -backend-config="key=tfstate/${application}/${envir}/${component}.json" 

terraform get

terraform $cmd

Here is how a typical run-tf.sh invocation looks like:

$ run-tf.sh application_name dev vpc plan

$ run-tf.sh application_name prod bastion apply