Iterate over nested data with for / for_each at resource level

Question

I am trying to work out how to iterate over nested variables from a complex object given in the following tfvars file using Terraform 0.12.10:

example.tfvars

virtual_network_data = {
  1 = {
    product_instance_id              = 1
    location                         = "somewhere"
    address_space                    = ["192.168.0.0/23"]
    dns_servers                      = []
    custom_tags                      = {"test":"test value"}
    subnets                          = [
      {
        purpose = "mgmt"
        newbits = 4
        item = 0
      },
      {
        purpose = "transit"
        newbits = 4
        item = 1
      }
    ]
  }
}

example.tf

variable "virtual_network_data" {} #Data comes from example.tfvars

variable "resource_group_name" {
    default = "my_resource_group"
}

variable "virtual_network_name" {
    default = "my_virtual_network"
}

####

resource "azurerm_subnet" "pool" {
    for_each             = var.virtual_network_data

    name                 = format("%s%s%02d", "subnet_", s.purpose, s.item)
    resource_group_name  = var.resource_group_name
    virtual_network_name = var.virtual_network_name
    address_prefix       = cidrsubnet(each.value["address_space"], s.newbits, s.item)

}

In example.tf I can use each.value["address_space"] to get to the top level variables, but I can't work out how to get to the items in subnets (s.purpose, s.item & s.newbits).

I have used dynamic blocks, as part of a parent resource (below), which works but in this case, I need to move the subnet into its own resource. Simply put, how do I get the first for_each to behave like the second for_each in the dynamic block?

resource "azurerm_virtual_network" "pool" {
  for_each                  = var.virtual_network_data

  name                      = format("%s%02d", local.resource_name, each.key)
  resource_group_name       = var.resource_group_name
  location                  = each.value["location"]
  address_space             = each.value["address_space"]
  dns_servers               = each.value["dns_servers"]
  tags                      = merge(local.tags, each.value["custom_tags"])

  dynamic "subnet" {
    for_each = [for s in each.value["subnets"]: {
      name   = format("%s%s%02d", "subnet_", s.purpose, s.item)
      prefix = cidrsubnet(element(each.value["address_space"],0), s.newbits, s.item)
    }]

    content {
      name            = subnet.value.name
      address_prefix  = subnet.value.prefix
    }
  }
}

Cheeky bonus, is there a way to replace s.item with something like each.key or count.index?

TIA

Answer 1

The technique in this situation is to use other Terraform language features to transform your collection to be a suitable shape for the for_each argument: one element per resource instance.

For nested data structures, you can use flatten in conjunction with two or more for expressions to produce a flat data structure with one element per nested object:

locals {
  network_subnets = flatten([
    for network_key, network in var.virtual_network_data : [
      for subnet in network.subnets : {
        network_key       = network_key
        purpose           = subnet.purpose
        parent_cidr_block = network.address_space[0]
        newbits           = subnet.newbits
        item              = subnet.item
      }
    ]
  ])
}

Then you can use local.network_subnets as the basis for repetition:

resource "azurerm_subnet" "pool" {
    # Each instance must have a unique key, so we'll construct one
    # by combining the network key, the subnet "purpose", and the "item".
    for_each = {
      for ns in local.network_subnets : "${ns.network_key}.${ns.purpose}${ns.item}" => ns
    }

    name                 = format("%s%s%02d", "subnet_", each.value.purpose, each.value.item)
    resource_group_name  = var.resource_group_name
    virtual_network_name = var.virtual_network_name
    address_prefix       = cidrsubnet(each.value.parent_cidr_block, each.value.newbits, each.value.item)
}

There's a similar example in the flatten documentation, as some additional context.