How to download a file from GitHub Enterprise using Terraform?

Question

Here is my s3_policy.json

{
  "Version":"2012-10-17",
  "Statement":[
    {
      "Sid":"mybucket",
      "Effect":"Allow",
      "Principal": "*",
      "Action":["s3:GetObject"],
      "Resource":[
        "arn:aws:s3:::${bucket_name}/*"
      ],
      "Condition": {
          "IpAddress": {
              "aws:SourceIp": [
              "10.xx.xxx.x",
              "172.168.xx.x",
              ........,
              .........,
              ..........,
              ...........,
              ]
          }
      }
    }
  ]
}

I have common repo which I use it for different projects. This common repo has a CIDR IP list in yaml format.

I would like to pull it into my Terraform project so that I can re use the same file instead of hardcoding IP addresses.

I'm unable to figure out a way to automate this instead of hardcoding IP addresses in this repo.

Answer 1

You could consume the IP addresses as a data source and use that instead.

Your policy document would then look like:

resource "aws_iam_policy" "whitelist_ips" {
  name        = "whitelist_ips"
  description = "${var.policy_description}"

  policy = <<EOF
{
  "Version":"2012-10-17",
  "Statement":[
    {
      "Sid":"mybucket",
      "Effect":"Allow",
      "Principal": "*",
      "Action":["s3:GetObject"],
      "Resource":[
        "arn:aws:s3:::${bucket_name}/*"
      ],
      "Condition": {
          "IpAddress": {
              "aws:SourceIp": ["${data.external.ip_addresses.result}"]
          }
      }
    }
  ]
}
EOF
}

You would need to create an external data source that can be ran that would fetch the IP addresses from some location and return the IPs as a comma separated string.

data "external" "ip_addresses" {
  program = ["python", "${path.module}/get_ips.py"]
}

where get_ips.py might look something like this:

#!/usr/bin/env python
from __future__ import print_function
import json
import re

yaml_string = """ - 1.2.3.4/32
 - 1.2.3.5/32
 - 1.3.0.0/16
"""

result = []
lines = yaml_string.split("\n")

for line in lines:
    # Remove empty lines
    if line != "":
        result.append(re.sub('\s*-\s*', '', line))

print(json.dumps(','.join(result)))

But obviously you need to go fetch the YAML list from Github instead of pointlessly hardcoding it in this data source.