I am running a spring boot service using spring data redis and here is the following configuration.
The service seems to work but I am seeing a stream of Lost Sentinel messages in the logs. The sentinel nodes are reachable form the VM where I am running the service. I was able to telnet to them directly from that VM. Any idea why this is happening?
spring:
profiles:
active: core-perf,swagger
default: core-perf,swagger
redis:
Pool: #Pool properties
# Max number of "idle" connections in the pool. Use a negative value to indicate
# an unlimited number of idle connections.
maxIdle: 8
# Target for the minimum number of idle connections to maintain in the pool.
# This setting only has an effect if it is positive.
minIdle: 0
# Max number of connections that can be allocated by the pool at a given time. Use a negative value for no limit.
maxActive: 8
# Maximum amount of time (in milliseconds) a connection allocation should block
# before throwing an exception when the pool is exhausted. Use a negative value
# to block indefinitely.
maxWait: -1
sentinel: #Redis sentinel properties.
master: mymaster
nodes: 10.202.56.209:26379, 10.202.56.213:26379, 10.202.58.80:26379
2015-06-15 17:30:54.896 ERROR 6677 --- [Thread-9] redis.clients.jedis.JedisSentinelPool : Lost connection to Sentinel at 10.202.58.80:26379. Sleeping 5000ms and retrying.
2015-06-15 17:30:59.894 ERROR 6677 --- [Thread-8] redis.clients.jedis.JedisSentinelPool : Lost connection to Sentinel at 10.202.56.213:26379. Sleeping 5000ms and retrying.
2015-06-15 17:30:59.897 ERROR 6677 --- [Thread-9] redis.clients.jedis.JedisSentinelPool : Lost connection to Sentinel at 10.202.58.80:26379. Sleeping 5000ms and retrying.
2015-06-15 17:31:04.975 ERROR 6677 --- [Thread-9] redis.clients.jedis.JedisSentinelPool : Lost connection to Sentinel at 10.202.58.80:26379. Sleeping 5000ms and retrying.
2015-06-15 17:31:04.976 ERROR 6677 --- [Thread-8] redis.clients.jedis.JedisSentinelPool : Lost connection to Sentinel at 10.202.56.213:26379. Sleeping 5000ms and retrying.
2015-06-15 17:31:09.976 ERROR 6677 --- [Thread-9] redis.clients.jedis.JedisSentinelPool : Lost connection to Sentinel at 10.202.58.80:26379. Sleeping 5000ms and retrying.
2015-06-15 17:31:09.976 ERROR 6677 --- [Thread-8] redis.clients.jedis.JedisSentinelPool : Lost connection to Sentinel at 10.202.56.213:26379. Sleeping 5000ms and retrying.
2015-06-15 17:31:15.054 ERROR 6677 --- [Thread-8] redis.clients.jedis.JedisSentinelPool : Lost connection to Sentinel at 10.202.56.213:26379. Sleeping 5000ms and retrying.
2015-06-15 17:31:15.055 ERROR 6677 --- [Thread-9] redis.clients.jedis.JedisSentinelPool : Lost connection to Sentinel at 10.202.58.80:26379. Sleeping 5000ms and retrying.
2015-06-15 17:31:20.055 ERROR 6677 --- [Thread-8] redis.clients.jedis.JedisSentinelPool : Lost connection to Sentinel at 10.202.56.213:26379. Sleeping 5000ms and retrying.
You can also use monitor to see what Sentinel is doing: $ redis-cli -p 6379 monitor OK 1546981642.808843 [0 127.0. 0.1:54765] "PING" 1546981642.866671 [0 127.0. 0.1:54765] "PUBLISH" "__sentinel__:hello" "127.0.
Commands. Use “redis-cli” with port 26379 to connect to sentinel. Note: you always want to tail the /var/log/sentinel/sentinel. log on all sentinels to see the cluster interaction.
Redis Sentinel is the high-availability solution for open-source Redis server. It provides monitoring of all Redis nodes and automatic failover should the master node become unavailable. This guide provides a sample configuration for a three-node Redis cluster.
We discovered the issue. There was a blank between the node pairs in the application.yml and once we removed this " " the Lost Sentinel log message disappeared.
so from
nodes: 10.202.56.209:26379, 10.202.56.213:26379, 10.202.58.80:26379
to
nodes: 10.202.56.209:26379,10.202.56.213:26379,10.202.58.80:26379
It would probably be a good thing if the committers looked at this as it would seem to be somewhat mysterious for users.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With