.NET Core - Hook incoming request with a Request-Id to outbound HTTP requests

Question

We are looking a way to HOOK a Request-Id (or a Correlation-Id) across multiple API requests as shown figure below:

The idea is to a have one single id to trace a particular request through multiple APIs in our Logs. We are using standard ILogger that comes out of the box with .NET Core 2.1.

What we have tried so far

• We have tried using Request-Id in the request header, which logs successfully but we are unable to retrieve the value to add it to the subsequent HTTP request to other APIs.

• We have noticed that there is also a CorrelationId that gets logged. However, we are not able to figure out how to update it.

• Also, we have noticed there is TraceIdentity available on HttpContextAccessor which could potentially solve our purpose. But, we do not know how to leverage it to solve our problem.

We cannot use Application Insights and want to rely on our own logging infrastructure framework. We cannot find anything much in the documentation.

Is there any out-of-box solution available which we can use without coming up with our own custom solution?

Answer 1

I raised the same question to @davidfowl on Twitter. He has replied with:

No there's nothing out of the box for this. There's an end to end that works with app insights but it isn't very fleshed out. You might consider just using the same middleware across teams. if There's a work item to address this in 3.0 https://github.com/aspnet/Hosting/issues/1350

So, from what it looks like that for now a custom middleware is the only way forward. This may change with future release.

Update

We ended-up creating a custom middleware as @DavidMcEleney had suggested. However, on top of it we added the CorrelationId to a AsyncLocal property. This helped us to access the CorrelationId anywhere in the code if/when required. Here is the code get/set CorrelationId:

using System;
using System.Threading;

public static class CorrelationContext
{
    private static readonly AsyncLocal<string> CorrelationId = new AsyncLocal<string>();

    public static void SetCorrelationId(string correlationId)
    {
        if (string.IsNullOrWhiteSpace(correlationId))
        {
            throw new ArgumentException(nameof(correlationId), "Correlation id cannot be null or empty");
        }

        if (!string.IsNullOrWhiteSpace(CorrelationId.Value))
        {
            throw new InvalidOperationException("Correlation id is already set");
        }

        CorrelationId.Value = correlationId;
    }

    public static string GetCorrelationId()
    {
        return CorrelationId.Value;
    }
}

To use in CorrelationMiddleware.cs

public class CorrelationMiddleware
{
    private readonly RequestDelegate _next;

    public CorrelationMiddleware(RequestDelegate next)
    {
        _next = next;
    }

    public async Task Invoke(HttpContext context)
    {
        context.Request.Headers.TryGetValue("Correlation-Id-Header", out var correlationIds);

        var correlationId = correlationIds.FirstOrDefault() ?? Guid.NewGuid().ToString();

        CorrelationContext.SetCorrelationId(correlationId);

        using (LogContext.PushProperty("Correlation-Id", correlationId))
        {
            await _next.Invoke(context);
        }
    }
}

If we need to access the CorrelationId in anywhere in the code later, then we simply call: CorrelationContext.GetCorrelationId();

Answer 2

I've been using Serilog for logging in dotnet core & have been pushing LogContext properties using middleware -

CorrelationMiddleware.cs

    public class CorrelationMiddleware
    {
    readonly RequestDelegate _next;

    public CorrelationMiddleware(RequestDelegate next)
    {
        _next = next;
    }

    public async Task Invoke(HttpContext context)
    {
        context.Request.Headers.TryGetValue(Headers.Correlation, out StringValues correlationId);

        using (LogContext.PushProperty("Correlation-Id", correlationId.FirstOrDefault()))
        {
            await _next.Invoke(context);
        }
    }
}

Register this in your Startup.cs Configure method:

app.UseMiddleware<CorrelationLogger>();

Then, when making an outbound http call - you can add a header to the HttpRequestMessage by adding

request.Headers.Add(Headers.Correlation, GetHeader("Correlation-Id"));

When searching logs, we can then search by the correlation id & see the full end to end between all APIs...

Answer 3

Not sure, if this is directly related, but we face same kind of issue for correlating request across apis and services in Service Fabric projects. We ended up setting and getting correlationId from CallContext (inside System.Runtime.Remoting.Messaging). Coincidentally just wrote something about it here. Please see if this helps.