Is it possible to reuse Terraform templates for different resources providing different values for variables?

Question

I am using Terraform to setup multiple droplets running Consul on DigitalOcean. Perhaps I am missing something basic, but it seems surprisingly difficult to provide right configuration for them.

resource "digitalocean_droplet" "prime" {
  count  = 3
  image  = "${data.digitalocean_image.prime.image}"
  name   = "${format("%s-%02d-%s", "prime", count.index + 1, var.region)}"
  private_networking = true

  # ...
}

Each machine has two network interfaces - public and private. With this setup it seems necessary to provide bind_addr pointing to private IP address of each droplet - otherwise consul exits with an error stating that there are multiple private (?!) addresses.

The most straight forward solution would be to provision each machine with a configuration file that in each case is almost the same, but with different value for bind_addr field, like that:

{
  "server": true,
  "data_dir": "/var/consul/data",
  "ui": true,
  "bind_addr": "${ private_ipv4_address }"
}

Isn't it what templates are for? I can't figure out how to use them that way. It seems that variables for a template can be provided only once, when the template is defined:

data "template_file" "consul-config" {
  template = "${file("templates/consul-config.json.tpl")}"

  vars {
    private_ipv4_address = "10.0.0.1" # At this point the real address is not known
  }
}

resource "digitalocean_droplet" "prime" {
  ...

  provisioner "file" {
    content = "${data.template_file.consul-config.rendered}"
    destination = "/etc/consul.d/server.json"

    # At this point the address is known: ${ self.private_ipv4_address },
    # but is it possible to pass it to the template?
  }
}

I tried to nest the data block in the resource block, but then I'm getting an error like that:

Error: resource 'digitalocean_droplet.prime' provisioner file (#7): unknown resource 'data.template_file.consul-config' referenced in variable data.template_file.consul-config.rendered

The work-around I use currently is to split the configuration into two parts (server and custom) and inline the contents of the custom in a file provisioner:

resource "digitalocean_droplet" "prime" {
  # ...

  provisioner "file" {
    content = "${data.template_file.consul-config.rendered}"
    destination = "/etc/consul.d/server.json"
  }

  # This is a custom configuration for each particular droplet
  provisioner "file" {
    content = "{ \"bind_addr\": \"${ self.ipv4_address_private }\", \"bootstrap\": ${ count.index == 0 }  }"
    destination = "/etc/consul.d/custom.json"
  }
}

It works, but the readability is impeded for several reasons:

1. All quotes have to be escaped

2. Everything has to be on single line (?)

3. No syntax highlighting or similar help from a text editor

Alternatively I considered using external program (like envsubst) to render the template or using built in format function together with file function, but each of those seems cumbersome.

Is there a straight forward way to achieve what I want?

Answer 1

Have you tried using writting a module?

This might be a good starting point: https://blog.gruntwork.io/how-to-create-reusable-infrastructure-with-terraform-modules-25526d65f73d

Answer 2

Templates are designed to take values from Terraform (whether from variable files, locals, data resources, etc.) and plug them into a templated file. Generally I've seen the resulting files be scripts which will be run by your eventual resource (an EC2 instance for AWS, a Droplet for DigitalOcean).

Modules (mentioned in another answer) are used to setup a collection of resources (say, a server, load balancer, some network resources) with a single entry point. I believe you're correct that it doesn't apply to your situation.

The way I'd accomplish something similar to what you want (get an IP address after the resource is created) is to template out a script and supply it to your droplet. Then have that script ask the droplet what its IP is. So you'd need to define a user_data resource in your droplet:

resource "digitalocean_droplet" "prime" {
  user_data = "${data.template_file.some_script.rendered}"
}

Then you also have a data template:

data "template_file" "consul-config" {
  template = "${file("templates/consul-config.json.tpl")}"

  vars {
    # Define Script Variables here, excluding IP
  }
}

which renders into a shell (or python, perl, whatever) script

# Get networking details
ifconfig | #pipe to function of your choice and format results as you please.
# Do stuff to configure server.