Helm Chart will install manually, will not install via Terraform

Question

I'm having issues getting the ingress-nginx Helm Chart to install via Terraform with Minikube, yet I'm able to install it successfully via the command line. Here is my vanilla Terraform code -

provider "kubernetes" {
  host                   = "https://127.0.0.1:63191"
  client_certificate     = base64decode(var.client_certificate)
  client_key             = base64decode(var.client_key)
  cluster_ca_certificate = base64decode(var.cluster_ca_certificate)
}

provider "helm" {
  kubernetes {

  }
}

resource "helm_release" "nginx" {
  name        = "beta-nginx"
  repository  = "https://kubernetes.github.io/ingress-nginx"
  chart       = "ingress-nginx"
  namespace   = "default"
}

I get the following logs when I apply the Terraform code above -

helm_release.nginx: Still creating... [4m31s elapsed]
2022-01-26T14:32:49.623-0600 [TRACE] dag/walk: vertex "root" is waiting for "provider[\"registry.terraform.io/hashicorp/helm\"] (close)"
2022-01-26T14:32:49.624-0600 [TRACE] dag/walk: vertex "meta.count-boundary (EachMode fixup)" is waiting for "helm_release.nginx"
2022-01-26T14:32:49.624-0600 [TRACE] dag/walk: vertex "provider[\"registry.terraform.io/hashicorp/helm\"] (close)" is waiting for "helm_release.nginx"
2022-01-26T14:32:51.299-0600 [INFO]  provider.terraform-provider-helm_v2.4.1_x5: 2022/01/26 14:32:51 [DEBUG] Service does not have load balancer ingress IP address: default/beta-nginx-ingress-nginx-controller: timestamp=2022-01-26T14:32:51.299-0600
2022-01-26T14:32:53.302-0600 [INFO]  provider.terraform-provider-helm_v2.4.1_x5: 2022/01/26 14:32:53 [DEBUG] Service does not have load balancer ingress IP address: default/beta-nginx-ingress-nginx-controller: timestamp=2022-01-26T14:32:53.302-0600
2022-01-26T14:32:54.626-0600 [TRACE] dag/walk: vertex "provider[\"registry.terraform.io/hashicorp/helm\"] (close)" is waiting for "helm_release.nginx"

Warning: Helm release "beta-nginx" was created but has a failed status. Use the `helm` command to investigate the error, correct it, then run Terraform again.

  with helm_release.nginx,
  on main.tf line 21, in resource "helm_release" "nginx":
  21: resource "helm_release" "nginx" {


Error: timed out waiting for the condition

  with helm_release.nginx,
  on main.tf line 21, in resource "helm_release" "nginx":
  21: resource "helm_release" "nginx" {

---

When I try installing the Helm Chart via the command line helm install beta-nginx ingress-nginx/ingress-nginx it installs the chart no problem.

Here are a few version numbers:

Terraform	1.0.5
Minikube	1.25.1
Kubernetes	1.21.7
Helm	3.7.2

Answer 1

This is because Terraform waits for LoadBalancer to get a public IP address, but this never happens, so the Error: timed out waiting for the condition error occurs:

$ kubectl get svc
NAME                                            TYPE           CLUSTER-IP       EXTERNAL-IP   PORT(S)                      AGE
beta-nginx-ingress-nginx-controller             LoadBalancer   <PRIVATE_IP>    <pending>     80:30579/TCP,443:30909/TCP   7m32s

You can install MetalLB to get a load-balancer implementation or create a NodePort instead of LoadBalancer. I'll briefly demonstrate the second option.

All you have to do is modify the controller.service.type value from the values.yaml file:

$ cat beta-nginx.tf 
provider "helm" {
  kubernetes {
    config_path = "~/.kube/config"
  }
}

resource "helm_release" "nginx" {
  name        = "beta-nginx"
  repository  = "https://kubernetes.github.io/ingress-nginx"
  chart       = "ingress-nginx"
  namespace   = "default"
  set {
    name  = "controller.service.type"
    value = "NodePort"
  }
}


$ terraform apply
...
      + set {
          + name  = "controller.service.type"
          + value = "NodePort"
        }
...
Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

$ kubectl get svc
NAME                                            TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                      AGE
beta-nginx-ingress-nginx-controller             NodePort    <PRIVATE_IP>     <none>        80:32410/TCP,443:31630/TCP   74s

As you can see above, the NodePort service has been created instead of the LoadBalancer.