StackExchange.Redis timeouts GET

Question

I am facing the issue with timeouts when using the StackExchange.Redis. Everything that I could find on stackoverflow I have tried.

Here is what I am getting: Timeout performing GET XXX, inst: 4, mgr: ProcessReadQueue, err: never, queue: 9, qu: 0, qs: 9, qc: 0, wr: 0, wq: 0, in: 65536, ar: 1, IOCP: (Busy=0,Free=1000,Min=400,Max=1000), WORKER: (Busy=35,Free=32732,Min=400,Max=32767)

In almost all exceptions that I get only queue and qs are changing and it had the value up to 11.

I am using nuget version 1.0.488. Application is hosted on IIS 8.5 on Windows Server 2012. On that machine we have 32gb of RAM and it is used from 30-50%. I have set the maxmemory to 16gb and the maxheap to 22gb. Saving to disk is disabled. I am using it only as a session storage.

Redis server(version x64-2.8.2402) is installed on the same machine. Here is the url for the redis-cli info that i have.

Also this is the part of exception that I am getting:

System.TimeoutException: Timeout performing GET Airports, inst: 1, mgr: ProcessReadQueue, err: never, queue: 4, qu: 0, qs: 4, qc: 0, wr: 0, wq: 0, in: 65536, ar: 1, IOCP: (Busy=0,Free=1000,Min=400,Max=1000), WORKER: (Busy=20,Free=32747,Min=400,Max=32767), clientName: OWNEROR-G875DJG   at StackExchange.Redis.ConnectionMultiplexer.ExecuteSyncImpl[T](Message message, ResultProcessor`1 processor, ServerEndPoint server) in c:\TeamCity\buildAgent\work\3ae0647004edff78\StackExchange.Redis\StackExchange\Redis\ConnectionMultiplexer.cs:line 1927    at StackExchange.Redis.RedisBase.ExecuteSync[T](Message message, ResultProcessor`1 processor, ServerEndPoint server) in c:\TeamCity\buildAgent\work\3ae0647004edff78\StackExchange.Redis\StackExchange\Redis\RedisBase.cs:line 80    at StackExchange.Redis.RedisDatabase.StringGet(RedisKey key, CommandFlags flags) in c:\TeamCity\buildAgent\work\3ae0647004edff78\StackExchange.Redis\StackExchange\Redis\RedisDatabase.cs:line 1451    at Resvoyage.Services.RedisCacheProvider.GetValue[T](String key) in c:\Development\thomalex\resvoyage\Resvoyage.Services\RedisCacheProvider.cs:line 88 

This is how my connection string is looking <add key="LocalRedis" value="127.0.0.1:6379,connectTimeout=25000,allowAdmin=true,password=xxx" />. ConnectionTimeout was 15000 and i tried incresing it to 25000

I have also increased minIoThread in machine.config for IIS. Here is the values that I have placed <processModel autoConfig="true" minIoThreads="50" minWorkerThreads="50" maxWorkerThreads = "100" maxIoThreads = "100"/>

Answer 1

Read redis slowlog to see if there is any slow queries on redis. Maybe you are using keys command? Redis is single threaded so one slow query could block everything and as the client will not wait for redis forever you run into timeouts.