System.Diagnostics.Activity is null in aspnet core 2.1

Question

We've just updated an aspnet core 2.0 application to 2.1 and have run into a problem with our usage/reliance on System.Diagnostics.Activity.

Background

We want a consistent 'correlation id' passed across our service boundaries, so that we can correlate log entries per request.

The approach we took was:

• Add a diagnostic listener to the middleware pipeline for Microsoft.AspNetCore.Hosting.HttpRequestIn.Start

• When this listener was invoked, check if there was a header in the context.request called "Request-Id", and if there wasn't then add one with a value of Activity.Current.Id

• Another piece of middleware took care of pushing the "Request-Id" header value into the logging context

This worked just fine in 2.0 and the hierarchical nature of Activity.Current.Id meant we could correlate log entries across different services and layers. In 2.1 we are now getting exceptions because Activity.Current appears to always be null on the point-of-entry for a request (i.e. the first service that is hit, in this case an API).

I've not managed to find any information that suggests that an activity is no longer automatically started whenever an HttpRequest comes in, but that's what it seems like is happening. Is anyone able to shed any light on what has changed and/or what we're doing wrong?

Some code

The startup configure method ...

public void Configure(IApplicationBuilder app, IHostingEnvironment env, DiagnosticListener diagnosticListener)
        {
           diagnosticListener.SubscribeWithAdapter(new HttpRequestInDiagnosticListener());
           app.UseMiddleware<SetRequestIdHeaderForHttpRequestInMiddleware>();

            if (env.IsDevelopment())
                app.UseDeveloperExceptionPage();
            else
                app.UseStatusCodePages();

            app.UseSecurityHeaders(Headers.AddSecurityHeaders());

            app.UseMvc();
        }

... and the custom classes involved

    public class HttpRequestInDiagnosticListener
    {
        [DiagnosticName("Microsoft.AspNetCore.Hosting.HttpRequestIn.Start")]
        public virtual void OnMiddlewareStarting(HttpContext httpContext)
        {
            Console.WriteLine($"Middleware Starting, path: {httpContext.Request.Path}");
        }
    }

    public class SetRequestIdHeaderForHttpRequestInMiddleware
    {
        private readonly RequestDelegate _next;
        private readonly DiagnosticSource _diagnostics;

        public SetRequestIdHeaderForHttpRequestInMiddleware(RequestDelegate next, DiagnosticSource diagnosticSource)
        {
            _next = next;
            _diagnostics = diagnosticSource;
        }

        public async Task Invoke(HttpContext context)
        {

            if (_diagnostics.IsEnabled("Microsoft.AspNetCore.Hosting.HttpRequestIn.Start"))
            {
                if (!context.Request.Headers.Keys.Contains("Request-Id"))
                {
                    context.Request.Headers.Add("Request-Id", Activity.Current.Id);
                }
            }

            await _next.Invoke(context);
        }
    }

Answer 1

Answering my own question, after days of looking into this, but succinctly YES: in 2.1 activities are no longer started in the same way that they were in 2.0. https://github.com/aspnet/Hosting/blob/release/2.1/src/Microsoft.AspNetCore.Hosting/Internal/HostingApplicationDiagnostics.cs#L54-L79

For an Activity to start you need to specifically be observing Microsoft.AspNetCore.Hosting.HttpRequestIn, whereas the code in the question was observing Microsoft.AspNetCore.Hosting.HttpRequestIn.Start

Unfortunately that isn't the only change in 2.1 that has affected the code in the question, and just changing the Activity Name being observed doesn't make this code work as 2.1 now handles Request-Id headers and CorrelationId log properties in its own way which interferes with this middleware code.