Use pre-installed Terraform plugins instead of downloading them with terraform init

Question

While running terraform init when using Terraform 0.11.3 we are getting the following error:

Initializing provider plugins... - Checking for available provider plugins on https://releases.hashicorp.com...

Error installing provider "template": Get https://releases.hashicorp.com/terraform-provider-template/: read tcp 172.25.77.25:53742->151.101.13.183:443: read: connection reset by peer.

Terraform analyses the configuration and state and automatically downloads plugins for the providers used. However, when attempting to download this plugin an unexpected error occured.

This may be caused if for some reason Terraform is unable to reach the plugin repository. The repository may be unreachable if access is blocked by a firewall.

If automatic installation is not possible or desirable in your environment, you may alternatively manually install plugins by downloading a suitable distribution package and placing the plugin's executable file in the following directory: terraform.d/plugins/linux_amd64

I realized it's because of connectivity issues with https://releases.hashicorp.com domain. For some obvious reasons, we will have to adjust with this connectivity issue as there are some SSL and firewall issues between the control server and Hashicorp's servers.

Is there any way we could bypass this by downloading the plugins from Hashicorp's servers and copying them onto the control server? Or any other alternative to avoid trying to download things from Hashicorp's servers?

Answer 1

You can use pre-installed plugins by either putting the plugins in the same directory as the terraform binary or by setting the -plugin-dir flag.

It's also possible to build a bundle of every provider you need automatically using the terraform-bundle tool.

I run Terraform in our CI pipeline in a Docker container so have a Dockerfile that looks something like this:

FROM golang:alpine AS terraform-bundler-build

RUN apk --no-cache add git unzip && \
    go get -d -v github.com/hashicorp/terraform && \
    go install ./src/github.com/hashicorp/terraform/tools/terraform-bundle

COPY terraform-bundle.hcl .

RUN terraform-bundle package terraform-bundle.hcl && \
    mkdir -p terraform-bundle && \
    unzip -d terraform-bundle terraform_*.zip

####################

FROM python:alpine

RUN apk add --no-cache git make && \
    pip install awscli

COPY --from=terraform-bundler-build /go/terraform-bundle/* /usr/local/bin/

Note that the finished container image also adds git, make and the AWS CLI as I also require those tools in the CI jobs that uses this container.

The terraform-bundle.hcl then looks something like this (taken from the terraform-bundle README):

terraform {
  # Version of Terraform to include in the bundle. An exact version number
  # is required.
  version = "0.10.0"
}

# Define which provider plugins are to be included
providers {
  # Include the newest "aws" provider version in the 1.0 series.
  aws = ["~> 1.0"]

  # Include both the newest 1.0 and 2.0 versions of the "google" provider.
  # Each item in these lists allows a distinct version to be added. If the
  # two expressions match different versions then _both_ are included in
  # the bundle archive.
  google = ["~> 1.0", "~> 2.0"]

  # Include a custom plugin to the bundle. Will search for the plugin in the 
  # plugins directory, and package it with the bundle archive. Plugin must have
  # a name of the form: terraform-provider-*, and must be build with the operating
  # system and architecture that terraform enterprise is running, e.g. linux and amd64
  customplugin = ["0.1"]
}