System.Exception: Correlation failed when posting back to /signin-oidc after migrating from app service to AKS

Question

The Problem

I'm encountering a 500 error when posting back to /signin-oidc for my ASP.NET Core application.

I have signin working as an app service behind an application gateway within Azure, but when I host it in a container in AKS behind the same gateway I get the error.

Looking at the container logs:

warn: Microsoft.AspNetCore.Authentication.OpenIdConnect.OpenIdConnectHandler[15]
      '.AspNetCore.Correlation.__PrN2tAGsRBPSGlBe4wQzX7rdufN534NuCeqjwUUEU' cookie not found.
fail: Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware[1]
      An unhandled exception has occurred while executing the request.
      System.Exception: An error was encountered while handling the remote login.
       ---> System.Exception: Correlation failed.
         --- End of inner exception stack trace ---
         at Microsoft.AspNetCore.Authentication.RemoteAuthenticationHandler`1.HandleRequestAsync()
         at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context)
         at Microsoft.AspNetCore.Localization.RequestLocalizationMiddleware.Invoke(HttpContext context)
         at Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware.<Invoke>g__Awaited|6_0(ExceptionHandlerMiddleware middleware, HttpContext context, Task task)

the main error seems to be "Correlation failed" with a warning about cookies not being found. I don't think the gateway is stripping cookies since it's working for the app service, so it's maybe something to do with AKS or the ingress controller?

The application has authentication enforced on all routes using the default policy as the fallback, with AddMicrosoftIdentityWepAppAuthentication doing the heavy lifting for the actual auth stuff.

// Configure AAD sign in
services.AddMicrosoftIdentityWebAppAuthentication(config);
services.AddAuthorization(options =>
{
    options.FallbackPolicy = options.DefaultPolicy;
    AuthorizationPolicies.Configure(options);
});
// When in prod we are running behind the application gateway
// so we need to override the RedirectUri to point to the correct url
// since the app doesn't know its url when running behind the gateway
if (!environment.IsDevelopment())
{
    services.Configure<OpenIdConnectOptions>(OpenIdConnectDefaults.AuthenticationScheme, options =>
    {
        options.Events = new OpenIdConnectEvents
        {
            OnRedirectToIdentityProvider = ctxt => {
                ctxt.ProtocolMessage.RedirectUri = cfg.RedirectUri;
                return Task.CompletedTask;
            }
        };
    });
}
...
app.UsePathBase("/ssp-pls");

I also tried to modify the cookie stuff like other answers suggest, but still no bueno.

app.UseCookiePolicy(new CookiePolicyOptions
{
    // HttpOnly =  HttpOnlyPolicy.Always,
    MinimumSameSitePolicy = SameSiteMode.None,
    Secure = CookieSecurePolicy.Always
    // MinimumSameSitePolicy = SameSiteMode.Lax
});

Here's the ingress

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ssp-ing
  annotations:
    nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
    nginx.ingress.kubernetes.io/affinity: cookie
spec:
  tls:
    - hosts:
        - selfserviceportal.beans.ca
  rules:
  - host: selfserviceportal.beans.ca
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: ssp-svc
            port:
              number: 80
  ingressClassName: nginx

Related links

• Correlation failed error in asp.net core 6.0 web application while authentication Android Management API scope
• Correlation failed. at Microsoft.AspNetCore.Authentication.RemoteAuthenticationHandler during OIDC authentication
• Correlation failed in net.core / asp.net identity / openid connect
• '.AspNetCore.Correlation....' cookie not found
• What could cause a correlation cookie to not be returned on specific devices
• Correlation failed in net.core / asp.net identity / openid connect
• OIDC login fails with 'Correlation failed' - 'cookie not found' while cookie is present
• Cannot set the Path for .AspNetCore.Cookies cookies
• How to handle "AspNetCore.Correlation.OpenIdConnect cookie not found" error on multiple browsers

Answer 1

Investigating

Lets look at the cookies for the postback on the working endpoint (appservice)

Then compare it to the broken one (AKS)

The broken one is missing the .AspNetCore.Correlation cookie, which is not surprising given the error message from the pod. If we check the show filtered out request cookies option we can get some more information

According to Edge, the cookie exists but is blocked. Comparing with the working example, the Path of the cookies is missing the basepath from the application where app.UsePathBase("/ssp-pls"); is being used.

Looking at the request, we are using the correct basepath in the postback.

The answer

It seems that this is happening because I'm visiting my application at the root / path without having the basepath /ssp-pls/ present in the URL. The error isn't happening with the appservice because the gateway rules ensure no requests arrive to the ASP.NET app without the basepath in the url.

Path	Backend
/ssp-pls/*	appservice running ASP.NET app
/*	static site

However with the AKS setup, the gateway looks like

Path	Backend
/*	AKS running ASP.NET app

This means that requests that don't include the basepath are still getting to the app.

If we visit /ssp-pls/ instead of / on the AKS app, the cookies are given the correct Path, so the signin works.

In this case it seems the best fix is ensuring that requests never make it to the app without the basepath already in the URL.

As mentioned, a bunch of other answers recommend changing global cookie configs. I tried removing the change we used to see if it was still necessary

- app.UseCookiePolicy(new CookiePolicyOptions
- {
-     // HttpOnly =  HttpOnlyPolicy.Always,
-     MinimumSameSitePolicy = SameSiteMode.None,
-     Secure = CookieSecurePolicy.Always
-     // MinimumSameSitePolicy = SameSiteMode.Lax
- });

but it gave some warnings and went back to failing without it, so I guess it is still necessary. Notably, after the change I couldn't find the .AspNetCore.Correlation cookie even after enabling show filtered out requesst cookies.

warn: Microsoft.AspNetCore.Http.ResponseCookies[1]
3
      The cookie '.AspNetCore.Correlation.3doj6oz452YzcAxJw3nzKm3z8LnVUb-AxQSqky5tOak' has set 'SameSite=None' and must also set 'Secure'.

You could try preventing the app from processing requests without the basepath but that seems inferior to just improving your routing so that you never hit the app without basepath in the first place.

Alternatively, you can apparently override the path when configuring the OpenIdConnectOptions, but I don't think this is the appropriate approach.

options.CorrelationCookie.Path = "/ssp-pls/signin-oidc"; // force basepath to be included