Issues with iterating over list, or set of elements in Terraform

Question

I am trying to launch AWS instances in different subnets created across AZs using the count variable over a list of subnet IDs, it fails with error messages with different approaches.

I have tried using element() and using [count.index], I am using TF v12.

Below is a snippet of the code, please ignore lines with #(commented).

resource "aws_instance" "workers" {
#count         =   length(data.terraform_remote_state.network.outputs.public_subnet_list)
count         = length(data.aws_subnet_ids.subnet_list.ids)
instance_type = var.worker_instance_type
ami           = var.k8_ami
key_name      = aws_key_pair.ssh_key.key_name
#subnet_id     = "${data.terraform_remote_state.network.outputs.public_subnet_list[count.index]}"
subnet_id = "${element(data.aws_subnet_ids.subnet_list.ids, count.index)}"
vpc_security_group_ids = [
aws_security_group.kubernetes.id
]
}

Below are two error snippets.

Error: Invalid index

on k8-cluster.tf line 85, in resource "aws_instance" "workers":
85:   subnet_id = "${data.aws_subnet_ids.subnet_list.ids[count.index]}"
|----------------
| count.index is 2
| data.aws_subnet_ids.subnet_list.ids is set of string with 4 elements

This value does not have any indices.

Another error with element():

Error: Error in function call

on k8-cluster.tf line 85, in resource "aws_instance" "workers":
85:   subnet_id = "${element(data.aws_subnet_ids.subnet_list.ids, count.index)}"
|----------------
| count.index is 3
| data.aws_subnet_ids.subnet_list.ids is set of string with 4 elements

Call to function "element" failed: cannot read elements from set of string.

Answer 1

The root problem here is that data.aws_subnet_ids.subnet_list.ids is a set value, rather than a list value, and so its elements are not in a particular order and therefore cannot be accessed by numeric index into a list.

To use it as a list requires deciding on how to order the elements. In this case it seems that the ordering isn't really important because the goal is just to create one instance per subnet, so passing the set to the sort function should be sufficient to sort them lexically:

resource "aws_instance" "workers" {
  count         = length(data.terraform_remote_state.network.outputs.public_subnet_list)
  instance_type = var.worker_instance_type
  ami           = var.k8_ami
  key_name      = aws_key_pair.ssh_key.key_name
  subnet_id     = sort(data.terraform_remote_state.network.outputs.public_subnet_list)[count.index]
  vpc_security_group_ids = [
    aws_security_group.kubernetes.id
  ]
}

---

In a future version of Terraform (not available at the time of writing in v0.12.0) a new for_each feature is planned that would make this more straightforward:

resource "aws_instance" "workers" {
  # Planned for a future Terraform release; not in v0.12.0
  for_each = data.terraform_remote_state.network.outputs.public_subnet_list

  instance_type = var.worker_instance_type
  ami           = var.k8_ami
  key_name      = aws_key_pair.ssh_key.key_name
  subnet_id     = each.value
  vpc_security_group_ids = [
    aws_security_group.kubernetes.id
  ]
}

An advantage of using for_each once it is implemented (aside from brevity) is that it will also tell Terraform to identify individual instances of this resource by the subnet id string, rather than by position in the list. That means that adding new subnets in future won't cause later instances to be "offset" and needlessly recreated, as would be true with my original example above.

Answer 2

This issue is happening because the returned data is in the form of a "SET", while the element is expecting the input in the form of a "LIST".

To resolve the issue you need to convert it into a list.

subnet_id = "${element(tolist(data.aws_subnet_ids.subnet_list.ids), count.index)}"

Answer 3

You just need to change the count like this:

count = "${length(data.aws_subnet_ids.subnet_list.ids)}"

And when you quote the variables, I suggest you do it like this:

"${var.var_name}" 

So the whole code will like below:

resource "aws_instance" "workers" {
    #count         =   "${length(data.terraform_remote_state.network.outputs.public_subnet_list)}"
    count         = "${length(data.aws_subnet_ids.subnet_list.ids)}"
    instance_type = "${var.worker_instance_type}"
    ami           = "${var.k8_ami}"
    key_name      = "${aws_key_pair.ssh_key.key_name}"
    #subnet_id     = "${data.terraform_remote_state.network.outputs.public_subnet_list[count.index]}"
    subnet_id = "${element(data.aws_subnet_ids.subnet_list.ids, count.index)}"
    vpc_security_group_ids = [
    "${aws_security_group.kubernetes.id}"
    ]
}

You can see the example in Terraform.