Redis too many open files error

Question

I am getting "too many open files error" when a certain number of users exceeds (its around 1200 concurrent users).

I increased the limit using this but I was getting same error.

Then I followed this and no change getting the same error.

For creating connection I am using in my django settings and using REDIS when I need it.

REDIS = redis.StrictRedis(host='localhost', port=6379, db=0)

Why I did it like that because it was suggested in redis mailing list like below:

a. create a global redis client instance and have your code use that.

Is that approach right for connection pooling? Or how I avoid this error of too many open files? In Django response I am getting

Connection Error (Caused by : [Errno 24] Too many open files)",),)'

Thanks.

Answer 1

You are creating a ConnectionPool per connection; depending on where you create the REDIS connection you might end up creating a new connection pool every time (eg. if its in view function).

You should make sure you create connections reusing an long lived connection pool; if you define the connection pool instance at module level and reuse that when you init connections you will be sure only 1 pool is created (one per python process at least).

If you see the "too many open files error" on Redis with ulimit set way higher than the amount of users (eg. ulimit 10k and 1k connections from django) than you might be doing something that leads to Redis connection leaking (and therefore not being closed for an amount of time).

I suggest you to start adding a connection pool and set a max connection limit there (its part of init signature); make sure the pool raises an exception only when the actual amount of connect users > than the limit.

If you can, increase the ulimit; Redis can easily take more than 1k connections.

If you really want to limit the amount of connections between your python scripts and Redis you should consider using the BlockingConnectionPool which will let clients wait when all connections are in use (rather than throw an exception) or perhaps use something like twemproxy in between.