Collection was modified; enumeration operation may not execute randomly popping up - HTTP?

Question

I have "Collection was modified; enumeration operation may not execute." exception populating randomly in my logs, it is not a persistent error but a really random and annoying one.

It seems to be originating from System.Net.Http? When adding headers but I am not sure exactly what is causing this error to pop up. I looked everywhere for this error and I understand the concept of the cause however I don't do any manual looping & editing of a list (except adding the headers) that could throw this exception?

I have a service that communicates to an external API using HTTPClient, this works most of the time however it sometimes fails with the above exception, I tried debugging this however I just can't seem to find a resolution! Hopefully, someone can spot the issue and point me in the right way.

Exception:

Source: System.Private.CoreLib 
 Message: Collection was modified; enumeration operation may not execute. 
 Stack trace:    at System.Collections.Generic.List`1.Enumerator.MoveNextRare()
   at System.Net.Http.Headers.HttpHeaders.AddHeaderInfo(HeaderDescriptor descriptor, HeaderStoreItemInfo sourceInfo)
   at System.Net.Http.Headers.HttpHeaders.AddHeaders(HttpHeaders sourceHeaders)
   at System.Net.Http.Headers.HttpRequestHeaders.AddHeaders(HttpHeaders sourceHeaders)
   at System.Net.Http.HttpClient.PrepareRequestMessage(HttpRequestMessage request)
   at System.Net.Http.HttpClient.SendAsync(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.PostAsync(Uri requestUri, HttpContent content, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.PostAsync(String requestUri, HttpContent content)
   at Hidden.Core.Service.BlaService.MyMethod(MyMethodModel model) in C:\Source\Hidden.Core\Service\BlaService.cs:line 168
   at Hidden.Web.Api.Controllers.ResourceController.MyMethod(MyMethodModel model) in C:\Source\Hidden.Web.Api\Controllers\ResourceController.cs:line 387
   at Microsoft.AspNetCore.Mvc.Internal.ActionMethodExecutor.TaskOfIActionResultExecutor.Execute(IActionResultTypeMapper mapper, ObjectMethodExecutor executor, Object controller, Object[] arguments)
   at System.Threading.Tasks.ValueTask`1.get_Result()
   at Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker.InvokeActionMethodAsync()
   at Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker.InvokeNextActionFilterAsync()
   at Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker.Rethrow(ActionExecutedContext context)
   at Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
   at Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker.InvokeInnerFilterAsync()
   at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.InvokeNextResourceFilter()
   at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.Rethrow(ResourceExecutedContext context)
   at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
   at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.InvokeFilterPipelineAsync()
   at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.InvokeAsync()
   at Microsoft.AspNetCore.Builder.RouterMiddleware.Invoke(HttpContext httpContext)
   at Hidden.Web.Api.Infrastructure.ExceptionHandlerMiddleware.Invoke(HttpContext context, IActionResultExecutor`1 executor) in C:\Source\Hidden.Web.Api\Infrastructure\ExceptionHandlerMiddleware.cs:line 31 
 Inner Exception: 

My Service:

private static readonly HttpClient client = new HttpClient();

public async Task<HttpResponseMessage> MyMethod(MyMethodModel model)
{
    //Escape chars
    model.Location = model.Location.Replace("'", "''");
    model.DirectoryName = model.DirectoryName.Replace("'", "''");

    //Set Endpoint
    var sharePointEndpoint = $"https://{model.Hostname}/sites/{model.Site}/_api/web/folders";

    //Set default headers
    client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", model.SharePointToken); //Set token
    client.DefaultRequestHeaders.Add("Accept", "application/json;odata=verbose");

    //Pre-Body data setup
    ResourceBodyMetaDataModel metaData = new ResourceBodyMetaDataModel() { type = "SP.Folder" };

    //Body data setup
    var bodyModel = new Dictionary<string, object>();
    bodyModel.Add("__metadata", metaData);
    bodyModel.Add("ServerRelativeUrl", $"{model.Location}/{model.DirectoryName}/");

    //Set content headers
    var strContent = new StringContent(JsonConvert.SerializeObject(bodyModel));
    strContent.Headers.ContentType = new MediaTypeHeaderValue("application/json");
    strContent.Headers.ContentType.Parameters.Add(new NameValueHeaderValue("odata", "verbose"));

    // Send request, grab response
    var response = await client.PostAsync(sharePointEndpoint, strContent); //Exception Points HERE

    //Return response message
    return response;
}

Answer 1

Overview:

It seemed that once I sent multiple requests to my API, these requests was conflicting with each other trying to overwrite the headers. I even tried clearing the headers first and then add them but yet I still encountered the above issue. The issue also started to include the exception: An item with the same key has already been added. Key: System.Net.Http.Headers.HeaderDescriptor.

Resolution:

To solve this I refactored my service to do the following:

//Setup request
var request = new HttpRequestMessage(HttpMethod.Post, sharePointEndpoint)
{
    Content = strContent
};

request.Headers.Authorization = new AuthenticationHeaderValue("Bearer", model.SharePointToken);
request.Headers.Add("Accept", "application/json;odata=verbose");
request.Headers.Add("X-HTTP-Method", "POST");
request.Headers.Add("ContentLength", bodyModelJson.Length.ToString());

// Send request, grab response
var response = await _httpClient.SendAsync(request);

In my case, the headers would depend on what operation I wanted to conduct. I removed the part where I was adding the default headers for HTTPClient and instead, added them within the request being made. As you can see, I now also used SendAsync();

I also injected HTTPClient as singleton. I had to initiate using my Constructor then afterwards whenever I want to use HTTPClient.

MyService:

private readonly HttpClient _httpClient;

public MyService(HttpClient httpClient)
{
    _httpClient = httpClient;
}

Startup.cs:

public void ConfigureServices(IServiceCollection services)
{
    /// SERVICES
    services.AddSingleton<HttpClient>();
}

Hopefully this helps other people experiencing this issue.