Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Recover from dropped connection in redis pub/sub

I am running client that is connecting to a redis db. The client is on a WiFi connection and will drop the connection at times. Unfortunately, when this happens, the program just keeps running without throwing any type of warning.

r = redis.StrictRedis(host=XX, password=YY...)
ps = r.pubsub()
ps.subscribe("12345")
for items in ps.listen():
    if items['type'] == 'message':
       data = items['data']

Ideally, what I am looking for is a catch an event when the connection is lost, try and reestablish the connection, do some error correcting, then get things back up and running. Should this be done in the python program? Should I have an external watchdog?

like image 269
Alexis Avatar asked Nov 30 '12 22:11

Alexis


People also ask

Does Redis pub/sub store messages?

Aside from data storage, Redis can be used as a Publisher/Subscriber platform. In this pattern, publishers can issue messages to any number of subscribers on a channel. These messages are fire-and-forget, in that if a message is published and no subscribers exists, the message evaporates and cannot be recovered.

How reliable is Redis pub sub?

The Redis pub/sub is not a reliable messaging system. Messages that are not retrieved will be discarded when your client is disconnected or a master/standby switchover occurs.

Is Redis pub/sub blocking?

Redis' pub/sub sends messages to clients subscribed (listening) on a channel. If you are not listening, you will miss the message (hence the blocking call).

Do we need to close Redis connection?

Continuously opening connections without closing is not a good practice. This will not only consume your resources but may also lead to program crash. The maximum number of file descriptors that you can open simultaneously is 1024.


1 Answers

Unfortunately, one have to 'ping' Redis to check if it is available. If You try to put a value to Redis storage, it will raise an ConnectionError exception if connection is lost. But the listen() generator will not close automatically when connection is lost.

I think that hacking Redis' connection pool could help, give it a try.

P.S. In is very insecure to connect to redis in an untrusted network environment.

like image 122
saabeilin Avatar answered Oct 21 '22 00:10

saabeilin