How to connect AWS Elasticache Redis cluster to Spring Boot app?

Question

I have Spring Boot app which connects to Redis cluster, using Jedis Connection Factory:

RedisClusterConfiguration redisClusterConfiguration = new RedisClusterConfiguration(redisProperties.getCluster().getNodes());
redisClusterConfiguration.setPassword(redisProperties.getPassword());
jedisConnectionFactory = new JedisConnectionFactory(redisClusterConfiguration);

and reading list of nodes from application.yml:

spring:
  redis:
    host: 127.0.0.1
    port: 6379
    timeout: 300s
    cluster:
      nodes: 127.0.0.1:6380,127.0.0.1:6381,127.0.0.1:6382

Now we want to switch to Elasticache since we are hosting our Redis cluster on AWS anyway. It would be done quite easily. If AmazonElastiCache lib could be used. Then we could just connect to Elasticache cluster with AWS credentials, pull available nodes put it in the list and pass it to Jedis instead hardcoding them in application.yml, like:

//get cache cluster nodes using AWS api
private List<String> getClusterNodes(){
    AmazonElastiCache client = AmazonElastiCacheClientBuilder.standard().withRegion(Regions.DEFAULT_REGION).build();
    DescribeCacheClustersRequest describeCacheClustersRequest = new DescribeCacheClustersRequest();
    describeCacheClustersRequest.setShowCacheNodeInfo(true);
    List<CacheCluster> cacheClusterList = client.describeCacheClusters(describeCacheClustersRequest).getCacheClusters();
    List<String> nodeList = new ArrayList<>();
    try {
        for (CacheCluster cacheCluster : cacheClusterList) {
            for(CacheNode cacheNode :cacheCluster.getCacheNodes()) {
                String nodeAddr = cacheNode.getEndpoint().getAddress() + ":" +cacheNode.getEndpoint().getPort();
                nodeList.add(nodeAddr);
            }
        }
    }
    catch(Exception e) {
        e.printStackTrace();
    }
    return nodeList;
}

But DevOps team said that they can't configure AWS access on all labs and they have reasons for it. Also instead of connecting to AWS and pulling all available clusters we need to connect to specific one by URL.

So I tried to pass Elasticache cluster url directly to Jedis as standalone and as a cluster in application.yml configuration. In both cases connection is established, but when App tries to write to Elasticache its gets MOVED exception:

org.springframework.web.util.NestedServletException: Request processing failed; nested exception is org.springframework.data.redis.ClusterRedirectException: Redirect: slot 1209 to 10.10.10.011:6379.; nested exception is redis.clients.jedis.exceptions.JedisMovedDataException: MOVED 1209 10.10.10.102:6379

Which as I understand means that App tried to write to one of the nodes in Elasticache, but wasn't able to connect.

So the question would be is there a way to connect to Elasticache Redis cluster from Spring Boot app using only Elasticache cluster URL?

I know that it's doable if Elasticache Memecache is used. Also Jedis driver is not a hard requirement.

Thank you.

Answer 1

After some research we learned that if AWS Elasticache cluster end-point is set as a node in RedisClusterConfiguration then driver (Jedis or Lettuce) is able to connect and find all the nodes in a Elasticache cluster. Also if one of the nodes goes down driver is able to communicate with Elasticache cluster through some other node.

We migrated to Lettuce driver while working on this upgrade as well, since Lettuce is default driver provided in Spring Boot Redis Started and supports latest Redis versions. Lettuce connections are designed to be thread-safe too, Jedis not.

Code example:

List<String> nodes = Collections.singletonList("****.***.****.****.cache.amazonaws.com:6379");
RedisClusterConfiguration clusterConfiguration = new RedisClusterConfiguration(nodes);
return new LettuceConnectionFactory(clusterConfiguration);

Answer 2

Inspired from Above Answer:, complete more detailed code

 List<String> nodes = Collections.singletonList("<cluster-host-name>:<port>");
 RedisClusterConfiguration clusterConfiguration = new RedisClusterConfiguration(nodes);
 
 ClusterTopologyRefreshOptions topologyRefreshOptions = ClusterTopologyRefreshOptions.builder().closeStaleConnections(true)
 .enableAllAdaptiveRefreshTriggers().build();
 
 ClusterClientOptions clusterClientOptions = ClusterClientOptions.builder().autoReconnect(true)
 .topologyRefreshOptions(topologyRefreshOptions).validateClusterNodeMembership(false)
 .build();
 //If you want to add tuning options
 LettuceClientConfiguration  lettuceClientConfiguration = LettuceClientConfiguration.builder().readFrom(ReadFrom.REPLICA_PREFERRED).clientOptions(clusterClientOptions).build();
 
 LettuceConnectionFactory lettuceConnectionFactory = new LettuceConnectionFactory(clusterConfiguration, lettuceClientConfiguration);
 lettuceConnectionFactory.afterPropertiesSet();//**this is REQUIRED**
 StringRedisTemplate redisTemplate = new StringRedisTemplate(lettuceConnectionFactory);