Concatenate two lists in Terraform 0.12 - concat()

Question

I want to concatenate two arrays in Terraform 0.12. In my example there are public and private subnets. I want to assign them both to the same network access lists. Following code shortened:

data "aws_subnet_ids" "private" {
    vpc_id = aws_vpc.main.id
    tags = {
        subnet-type  = "private"
    }
}

data "aws_subnet_ids" "public" {
    vpc_id = aws_vpc.main.id
    tags = {
        subnet-type  = "public"
    }
}


resource "aws_network_acl" "networks" {
    vpc_id = aws_vpc.main.id

    subnet_ids = concat(data.aws_subnet_ids.private.ids, data.aws_subnet_ids.public.ids)
    [...]
}

If I use the following outputs:

output "private_subnets" {
  value = data.aws_subnet_ids.private.ids
}

output "public_subnets" {
  value = data.aws_subnet_ids.public.ids
}

The following output is generated:

private_subnets = [
  "subnet-243zr427rhhfjseb9",
  "subnet-we789rh2438fchb6e",
  "subnet-092rz7g82fhhkui74",
]
public_subnets = [
  "subnet-12230qegvg764e9d",
  "subnet-123465svgvgf0d7e",
]

So everything should work. But the following error is given:

iptizer@machine:~/src/infra$ terraform12 apply
[...]

Error: Invalid function argument

  on nacls.tf line 19, in resource "aws_network_acl" "networks":
  19:     subnet_ids = concat(data.aws_subnet_ids.private.ids, data.aws_subnet_ids.public.ids)
    |----------------
    | data.aws_subnet_ids.private.ids is set of string with 3 elements

Invalid value for "seqs" parameter: all arguments must be lists or tuples; got
set of string.

Bug.. or what am I missing?

Answer 1

Terraform 0.12 makes a stronger distinction between list and set values than 0.11 did, and includes some additional checks like this.

In this particular case, concat is failing in this way because concatenation requires all of the elements to have a well-defined order so that the result can also have a well-defined order. Sets are not ordered, so this check is in place to remind you to explicitly select a suitable ordering when converting to list, or to not convert to list at all.

In this particular case it doesn't seem that the ordering is particularly important, so the lexical ordering implemented by sort could be sufficient:

  subnet_ids = concat(
    sort(data.aws_subnet_ids.private.ids),
    sort(data.aws_subnet_ids.public.ids),
  )

(Because the conversion from list set of string to list of string also imposes lexical ordering, this is functionally equivalent to tolist for string sets. I generally prefer sort here because it's a clue to a future reader that the result will be in lexical order.)

Another option is to say in the world of sets, and use setunion instead:

  subnet_ids = setunion(
    data.aws_subnet_ids.private.ids,
    data.aws_subnet_ids.public.ids,
  )

Since there should be no duplicates between these two lists it doesn't really matter which approach you use here, but for completeness I'll note that in the event that both of these sets contained the same subnet id the setunion operation would dedupe them, because each unique value can only appear zero or one times in a set.

---

At the time I write this, where count is still the dominant way to create one resource instance per item in a collection, converting to a list eventually is commonly required so that the individual instances can have a required order. Once for_each is implemented, there will be an advantage to using sets rather than lists in situations like this:

resource "aws_instance" "per_subnet_example" {
  # resource-level for_each is not implemented at the time of writing,
  # but planned for a future release.
  for_each = setunion(
    data.aws_subnet_ids.private.ids,
    data.aws_subnet_ids.public.ids,
  )

  # ...
}

When using for_each over a set instead of count, Terraform will identify each instance by the value from the set rather than by consecutive indexes, so one instance from this resource might have the address aws_instance.per_subnet_example["subnet-abc123"], and that means that when elements are added and removed from that set Terraform can just create/destroy the corresponding individual instance rather than potentially recreating everything after the change in the ordered sequence.

Terraform providers are using sets like this in places where it makes sense in order to make this for_each pattern easier to use once it arrives, but unfortunately that means we need to write some extra explicit type conversions in the meantime in order to be explicit that we're working with these values in an sequence-like way rather than a set-like way.