Split terraform tfstate file

Question

We started off with one tfstate file, and over time it's grown quite a bit.

Now, it's really slow to plan and I'd now like to split out into several tfstate files (one for our development environment, one for general shared infrastructure, and one per production environment).

Like how it's described at https://charity.wtf/2016/03/30/terraform-vpc-and-why-you-want-a-tfstate-file-per-env/ and Terraform Multiple State Files Best Practice Examples.

Is there any existing tool (built-in or not) to help with this? Sort of like terraform state mv but between tfstates?

Answer 1

terraform state mv has the -state-out flag, where you can define another state to move your resources into.

However, I couldn't get it to work with version 0.11.14, thus I manually cut & pasted the modules from the state file I needed to move to the other state, which worked brilliantly.

EDIT: Here's a workaround, which basically dwonloads both statefiles, moves the states desired, and then reuploads them to the S3 buckets: https://github.com/hashicorp/terraform/pull/15652#issuecomment-410754814

Answer 2

Having a separate state file each for infrastructure and application makes sense.

Is there any existing tool (built-in or not) to help with this? Sort of like terraform state mv but between tfstates?

No, as far as I can tell. Move out shared parts (e.g. ECS clusters, ALB, network configuration, iam roles etc.) into a separate project/repository.

When using S3 as state backend you can define different paths for your infrastructure and application states, for example:

• /infrastructure/nonprod/terraform.tfstate
• /infrastructure/prod/terraform.tfstate
• /apps/app1/test/terraform.tfstate
• /apps/app1/uat/terraform.tfstate
• /apps/app1/prod/terraform.tfstate

When you want to deploy your application to TEST or UAT you simply call terraform init before terraform apply in your infrastructure project by providing the path to your non-prod S3 state. Then call terraform init on your app terraform config by providing the path to your TEST or UAT path.

Ideally, you can create your own shell scripts to provision and deploy your apps. Then in your favourite CI you can create a pipeline to provision infrastructure and deploy apps as you wish. Make sure you parameterize those scripts so you can pass which environment you want to provision or which app you want to deploy, for example:

./my-shared-infrastructure/provision-infrastructure.sh nonprod

./my-app-1/deploy-application.sh uat v1.0