.NET Core HttpContext.Request.Headers.Remove() doesn't work unless the Headers are enumerated first

Question

I try to remove a header from the httpContext.Request.Headers by running the following statement:

httpContext.Request.Headers.Remove("HeaderName");

This doesn't do anything, the header is still present in the IHeaderDictionary. After trying for a while I noticed that when I convert the IHeaderDioctionary to a List first, the header is successfully removed.

_ = httpContext.Request.Headers.ToList();
httpContext.Request.Headers.Remove("HeaderName");

Another way to do it is to first check if the key exists. the .ContainsKey() method enumerates the IHeaderDictionary, which makes the .Remove() work.

if (context.Request?.Headers?.ContainsKey("HeaderName") ?? false)
{
  context.Request.Headers.Remove("HeaderName");
}

Now while my code runs and does what I want it to do, I wonder why it is necessary to first enumerate the IHeaderDictionary? And is there a better way to achieve this?

To add some more context, here's how I pass the httpContext to my method in the Startup.cs:

// Startup.cs
public void ConfigureServices(IServiceCollection services)
{
    services
        .AddAuthentication(options =>
        {
            options.DefaultScheme = "default";
            options.DefaultChallengeScheme = "default";
        })
        .AddPolicyScheme("default", "default", options =>
        {
            options.ForwardDefaultSelector = context =>
            {
                // in this async method I remove the header
                return schemeService.GetSchemeAsync(context).Result;
            };
        })
}

Answer 1

This is an old question, but I've experienced the same issue recently so just posting the answer for anybody who may face it too.

The problem I had was in a custom .NET Core middleware for removing an HTTP header from HttpContent.Request.Headers. It worked correctly locally and on a Linux app service in Azure, but did not work on a Windows app service. Both app services were configured to use .NET Core 7.

Middleware code was as simple as that, note that the headers collection is not enumerated before calling the .Remove() method:

public async Task InvokeAsync(HttpContext httpContext)
{
   httpContext?.Request.Headers.Remove("HeaderName");

   await _next(httpContext);
}

The root cause of this issue is the method IHeaderDictionary.Remove() that behaves differently in Kestrel and HTTP.sys environments prior to .NET 8.0:

• Kestrel always loads all headers and the removal operation works just fine without any collection enumerations
• HTTP.sys lazy-loads the headers collection and it causes a side-effect for headers with default values (Connection, Accept-Language, Accept-Encoding, etc.): if header value is not explicitly loaded before an attempt to remove it, then it is skipped and nothing gets removed. This is why calling .ContainsKey(), Headers["HeaderName"] or .ToList() as described in the question "resolves" the problem - it just makes HTTP.sys to load the real header value and then all conditions inside .Remove() are passed and the header is removed correctly.

Here is the related GitHub issue from aspnetcore repository with detailed explanation: https://github.com/dotnet/aspnetcore/issues/43230. The issue has been fixed in .NET 8.0 and newer.