Tune Redis connection inside an Azure Function to prevent timeouts

Question

TL;DR

How does one amend the min number of threads for redis within an Azure Function?

Problem

I have an Azure Function that uses redis (via StackExchange.Redis package) to cache some values, or retrieve the existing value if already exists. I'm currently getting timeout issues that look to be because the Busy IOCP threads exceeds the Min IOCP thread value.

2016-09-08T11:52:44.492 Exception while executing function: Functions.blobtoeventhub. mscorlib: Exception has been thrown by the target of an invocation. StackExchange.Redis: Timeout performing SETNX 586:tag:NULL, inst: 1, mgr: Inactive, err: never, queue: 4, qu: 0, qs: 4, qc: 0, wr: 0, wq: 0, in: 260, ar: 0, clientName: RD00155D3AE265, IOCP: (Busy=8,Free=992,Min=2,Max=1000), WORKER: (Busy=7,Free=32760,Min=2,Max=32767), Local-CPU: unavailable (Please take a look at this article for some common client-side issues that can cause timeouts: https://github.com/StackExchange/StackExchange.Redis/tree/master/Docs/Timeouts.md).

According to the docs on timeouts, the resolution involves adjusting the MinThread count:

How to configure this setting:

In ASP.NET, use the "minIoThreads" configuration setting under the configuration element in machine.config. If you are running inside of Azure WebSites, this setting is not exposed through the configuration options. You should be able to set this programmatically (see below) from your Application_Start method in global.asax.cs. Important Note: the value specified in this configuration element is a per-core setting. For example, if you have a 4 core machine and want your minIOThreads setting to be 200 at runtime, you would use . Outside of ASP.NET, use the ThreadPool.SetMinThreads(…) API.

In an Azure Function a global.asax.cs file is not available, and the use of ThreadPool.SetMinThreads has little information associated with it I can parse! There is a similar question on webjobs that is unanswered.

My specifics

• Redis = Azure Redis Cache Standard 1Gb
• Azure Function = version 0.5
  • StackExchange.Redis = version 1.1.603

Redis code is in a separate file to main function.

using StackExchange.Redis;
using System.Text;

private static Lazy<ConnectionMultiplexer> lazyConnection = new Lazy<ConnectionMultiplexer>(() =>
        {
            string redisCacheName = System.Environment.GetEnvironmentVariable("rediscachename", EnvironmentVariableTarget.Process).ToString();;
            string redisCachePassword = System.Environment.GetEnvironmentVariable("rediscachepassword", EnvironmentVariableTarget.Process).ToString();;
            return ConnectionMultiplexer.Connect(redisCacheName + ",abortConnect=false,ssl=true,password=" + redisCachePassword);
        });
        
public static ConnectionMultiplexer Connection
{
    get
    {
        return lazyConnection.Value;
    }
}

static string depersonalise_value(string input, string field, int account_id)
{
    IDatabase cache = Connection.GetDatabase();
    string depersvalue = $"{account_id}:{field}:{input}";
    string value = $"{account_id}{Guid.NewGuid()}";
    bool created = cache.StringSet(depersvalue, value, when: When.NotExists);
    string retur = created? value : cache.StringGet(depersvalue).ToString();
    return (retur);
}

Answer 1

Ultimately, we needed to pursue @mathewc's answer and added a line into the connection multiplexer code to set the min threads to 500

readonly static Lazy<ConnectionMultiplexer> lazyConnection =

    new Lazy<ConnectionMultiplexer>(() =>

    {
      ThreadPool.SetMinThreads(500, 500);

Additionally, further tuning was required and the code enhanced via a SO code review. The main importance here, is the drastically upping of timeouts.

using StackExchange.Redis;
using System.Text;
using System.Threading;

readonly static Lazy<ConnectionMultiplexer> lazyConnection =
    new Lazy<ConnectionMultiplexer>(() =>
    {
        ThreadPool.SetMinThreads(500, 500);
        string redisCacheName = System.Environment.GetEnvironmentVariable("rediscache_name", EnvironmentVariableTarget.Process).ToString();
        string redisCachePassword = System.Environment.GetEnvironmentVariable("rediscache_password", EnvironmentVariableTarget.Process).ToString();
        return ConnectionMultiplexer.Connect(new ConfigurationOptions
        {
            AbortOnConnectFail = false,
            Ssl = true,
            ConnectRetry = 3,
            ConnectTimeout = 5000,
            SyncTimeout = 5000,
            DefaultDatabase = 0,
            EndPoints = { { redisCacheName, 0 } },
            Password = redisCachePassword
        });
    });


public static ConnectionMultiplexer Connection => lazyConnection.Value;


static string depersonalise_value(string input, string field, int account_id)
{
    IDatabase cache = Connection.GetDatabase();
    string depersvalue = $"{account_id}:{field}:{input}";
    string existingguid = (string)cache.StringGet(depersvalue);
    if (String.IsNullOrEmpty(existingguid)){ 
        string value = $"{account_id}{Guid.NewGuid()}";
        cache.StringSet(depersvalue, value);
        return value;
    }
    return existingguid;
}