Error creating CacheSecurityGroup: InvalidParameterValue

Question

I am trying use Terraform to spin up a Redis instance on Elasticache and am running into the following error.

* module.main.module.redis.aws_elasticache_security_group.redis: 1 error(s) occurred:

* aws_elasticache_security_group.redis: Error creating CacheSecurityGroup: InvalidParameterValue: Use of cache security groups is not permitted in this API version for your account.

Nothing I've found in GH issues has been helpful. Here is what my Terraform looks like this (I've confirmed that the variables are being passed in correctly):

resource "aws_elasticache_subnet_group" "redis" {
  name       = "vpc-public-subnet"
  description = "subnet where redis will live"
  subnet_ids = ["${var.subnet}"]
}

resource "aws_elasticache_security_group" "redis" {
  name                 = "redis-security-group"
  security_group_names = ["${var.redis_sec_group}"]
}

resource "aws_elasticache_replication_group" "redis" {
  automatic_failover_enabled    = true
  availability_zones            = ["us-east-2a"]
  replication_group_id          = "${var.environment}-myapp-rep-group-1"
  replication_group_description = "redis rep group - ${var.environment} env"
  node_type                     = "cache.t2.micro"
  number_cache_clusters         = 2
  parameter_group_name          = "default.redis3.2"
  port                          = 6379
  at_rest_encryption_enabled    = true
  transit_encryption_enabled    = true

  subnet_group_name = "${aws_elasticache_subnet_group.redis.name}"
  security_group_ids = ["${aws_elasticache_security_group.redis.id}"]

  lifecycle {
    ignore_changes = ["number_cache_clusters"]
  }
}

resource "aws_elasticache_cluster" "redis" {
  cluster_id           = "${var.environment}-myapp"
  count                = 1
  replication_group_id = "${aws_elasticache_replication_group.redis.id}"
}

I thought the issue may be with my IAM User, so I added the AmazonElastiCacheFullAccess policy but it's still saying it is not permitted. I did that after reading a post on AWS docs about API_CreateCacheSecurityGroup and confirmed those three policies are included in AmazonElastiCacheFullAccess.

There seems to be some buggy behavior around these resources

https://github.com/hashicorp/terraform/issues/10127

---

My Solution

Sorry, please bear with me here. Posting this and completely writing it out did help me process my thoughts. I found that the aws_elasticache_security_group were unnecessary and just decided to pass ["${var.redis_sec_group}"] directly into security_group_ids for aws_elasticache_replication_group.

This may seem obvious for someone who has already dealt with this before, and now to me it does as well. But coming into this brand new it wasn't. So this isn't a solution to that permissions issue I was getting. But, like with many things, I took a step back and questioned whether or not I actually needed it, and that answer was no.

Answer 1

It looks like you've already realized that the aws_elasticache_security_group is only for use in EC2 classic accounts when you don't use VPCs. More recently created accounts don't allow the creation of network level resources (instances, load balancers, RDS instances, Elasticache instances etc) outside of a VPC.

This is mentioned in the Terraform docs for the aws_elasticache_security_group resource:

NOTE: ElastiCache Security Groups are for use only when working with an ElastiCache cluster outside of a VPC. If you are using a VPC, see the ElastiCache Subnet Group resource.

The AWS docs for Elasticache security groups go into further detail:

Important

Amazon ElastiCache security groups are only applicable to clusters that are not running in an Amazon Virtual Private Cloud environment (VPC). If you are running in an Amazon Virtual Private Cloud, Security Groups is not available in the console navigation pane.

If you are running your ElastiCache nodes in an Amazon VPC, you control access to your clusters with Amazon VPC security groups, which are different from ElastiCache security groups. For more information about using ElastiCache in an Amazon VPC, see Amazon VPCs and ElastiCache Security