How to set node taints using Terraform for Amazon EKS

Question

I am building an AWS EKS cluster using this Terraform provider. However, I can't find a way to apply node taints to managed node groups or worker groups. This issue and its resolution seem to suggest that this is not possible. Is there any way to do this?

Answer 1

In the Terraform script for that provider, you can add the following to a worker group:

(in main.tf)

worker_groups = [
  {
      name                          = "my_node_group"
      instance_type                 = "t3.medium"
      asg_desired_capacity          = 1
      asg_min_size                  = 1
      additional_security_group_ids = [aws_security_group.all_worker_mgmt.id]
      kubelet_extra_args            = "--node-labels=my_node_label --register-with-taints=my_node_label:NoSchedule"
      asg_max_size                  = 1
      tags = []
    },
]

The important part is to set kubelet_extra_args to apply a node label to the node and to use that node label to set the taint using --register-with-taints. These are commands run by EKS on each worker node at startup. (Note that all the other parameters I have set in the worker group can be changed based on your requirement)

You can check taints on nodes by using kubectl describe node <node_ip>.

Answer 2

Now eks module does support taints out of the box, just use the following configuration:

module "eks" {
  source = "terraform-aws-modules/eks/aws"
  ...
  node_groups = {
    test_name = {
      ...
      taints = [
        {
          key    = "dedicated"
          value  = "statefulset"
          effect = "NO_SCHEDULE"
        }
      ] 
    }
  }
}

Answer 3

Notice that you don't have to use a dedicated module for this feature because Hashicorp AWS provider supports taints out of the box with the taint configuration block:

resource "aws_eks_node_group" "statefulset-ng" {
  cluster_name    = aws_eks_cluster.main.name
  node_group_name = "statefulset-ng"
  .
  .
  .
  
  # Block 1 
  taint {
    key = "statefulset-no-schedule"
    value  = "true"
    effect = "NO_SCHEDULE"
  }

  # Block 2    
  taint {
    key = "statefulset-no-execute"
    value  = "true"
    effect = "NO_EXECUTE"
  }
}

Notice that each taint needs to be configured in a sperated configuration block.

Answer 4

This is how I created my node group with a taint using aws_eks_node_group resource with a pre-created EKS cluster.

resource "aws_eks_node_group" "test" {
  cluster_name    = var.cluster_name
  node_group_name = "test"
  node_role_arn   = master.worker_iam_role_arn
  subnet_ids      = var.vpc.private_subnets_id
  disk_size       = 20
  taint {
    key = "dedicated"
    value = "gpuGroup"
    effect = "NO_SCHEDULE"
  }
  scaling_config {
      desired_size = 1
      max_size     = 3
      min_size     = 1
  }
  labels = {
        "some-labels" = "labels"
  }
  instance_types = ["t3.micro"]
  remote_access {
    ec2_ssh_key = ssh-key.key_name
  }
}

Reference: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/eks_node_group#taint