Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Getting a lost Sentinel error message for Redis

I am running a spring boot service using spring data redis and here is the following configuration.

The service seems to work but I am seeing a stream of Lost Sentinel messages in the logs. The sentinel nodes are reachable form the VM where I am running the service. I was able to telnet to them directly from that VM. Any idea why this is happening?

spring:
  profiles:
    active: core-perf,swagger
    default: core-perf,swagger
  redis:
    Pool:  #Pool properties
      # Max number of "idle" connections in the pool. Use a negative value to indicate
      # an unlimited number of idle connections.
      maxIdle: 8
      # Target for the minimum number of idle connections to maintain in the pool.
      # This setting only has an effect if it is positive.
      minIdle: 0
      # Max number of connections that can be allocated by the pool at a given time. Use a negative value for no limit.
      maxActive: 8
      # Maximum amount of time (in milliseconds) a connection allocation should block
      # before throwing an exception when the pool is exhausted. Use a negative value
      # to block indefinitely.
      maxWait: -1
    sentinel: #Redis sentinel properties.
      master: mymaster
      nodes: 10.202.56.209:26379, 10.202.56.213:26379, 10.202.58.80:26379

2015-06-15 17:30:54.896 ERROR 6677 --- [Thread-9] redis.clients.jedis.JedisSentinelPool    : Lost connection to Sentinel at  10.202.58.80:26379. Sleeping 5000ms and retrying.
2015-06-15 17:30:59.894 ERROR 6677 --- [Thread-8] redis.clients.jedis.JedisSentinelPool    : Lost connection to Sentinel at  10.202.56.213:26379. Sleeping 5000ms and retrying.
2015-06-15 17:30:59.897 ERROR 6677 --- [Thread-9] redis.clients.jedis.JedisSentinelPool    : Lost connection to Sentinel at  10.202.58.80:26379. Sleeping 5000ms and retrying.
2015-06-15 17:31:04.975 ERROR 6677 --- [Thread-9] redis.clients.jedis.JedisSentinelPool    : Lost connection to Sentinel at  10.202.58.80:26379. Sleeping 5000ms and retrying.
2015-06-15 17:31:04.976 ERROR 6677 --- [Thread-8] redis.clients.jedis.JedisSentinelPool    : Lost connection to Sentinel at  10.202.56.213:26379. Sleeping 5000ms and retrying.
2015-06-15 17:31:09.976 ERROR 6677 --- [Thread-9] redis.clients.jedis.JedisSentinelPool    : Lost connection to Sentinel at  10.202.58.80:26379. Sleeping 5000ms and retrying.
2015-06-15 17:31:09.976 ERROR 6677 --- [Thread-8] redis.clients.jedis.JedisSentinelPool    : Lost connection to Sentinel at  10.202.56.213:26379. Sleeping 5000ms and retrying.
2015-06-15 17:31:15.054 ERROR 6677 --- [Thread-8] redis.clients.jedis.JedisSentinelPool    : Lost connection to Sentinel at  10.202.56.213:26379. Sleeping 5000ms and retrying.
2015-06-15 17:31:15.055 ERROR 6677 --- [Thread-9] redis.clients.jedis.JedisSentinelPool    : Lost connection to Sentinel at  10.202.58.80:26379. Sleeping 5000ms and retrying.
2015-06-15 17:31:20.055 ERROR 6677 --- [Thread-8] redis.clients.jedis.JedisSentinelPool    : Lost connection to Sentinel at  10.202.56.213:26379. Sleeping 5000ms and retrying.
like image 877
EvilJinious1 Avatar asked Jun 15 '15 21:06

EvilJinious1


People also ask

How do I know if Redis Sentinel is running?

You can also use monitor to see what Sentinel is doing: $ redis-cli -p 6379 monitor OK 1546981642.808843 [0 127.0. 0.1:54765] "PING" 1546981642.866671 [0 127.0. 0.1:54765] "PUBLISH" "__sentinel__:hello" "127.0.

How do I connect to Redis Sentinel?

Commands. Use “redis-cli” with port 26379 to connect to sentinel. Note: you always want to tail the /var/log/sentinel/sentinel. log on all sentinels to see the cluster interaction.

What is Redis Sentinel service?

Redis Sentinel is the high-availability solution for open-source Redis server. It provides monitoring of all Redis nodes and automatic failover should the master node become unavailable. This guide provides a sample configuration for a three-node Redis cluster.


1 Answers

We discovered the issue. There was a blank between the node pairs in the application.yml and once we removed this " " the Lost Sentinel log message disappeared.

so from

nodes: 10.202.56.209:26379, 10.202.56.213:26379, 10.202.58.80:26379

to

nodes: 10.202.56.209:26379,10.202.56.213:26379,10.202.58.80:26379

It would probably be a good thing if the committers looked at this as it would seem to be somewhat mysterious for users.

like image 88
EvilJinious1 Avatar answered Sep 18 '22 11:09

EvilJinious1