Unable to retrieve document from: 'https://ids.com/.well-known/openid-configuration'

Question

I've been developing multiple applications that rely on Identity Server 4(IDS4) for authentication using OIDC. Everything worked great till I put the applications behind a proxy using SSL-offloading.

The goal is to be able to visit a site. When you request to login it should then redirect you to IDS4 validate you, then send you back. This is standard..

What really happens. 403 Error:

An unhandled exception occurred while processing the request.
HttpRequestException: Response status code does not indicate success: 403 (Forbidden).
System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()
IOException: IDX20804: Unable to retrieve document from: 'https://ids.com/.well-known/openid-configuration'.
Microsoft.IdentityModel.Protocols.HttpDocumentRetriever.GetDocumentAsync(string address, CancellationToken cancel)
InvalidOperationException: IDX20803: Unable to obtain configuration from: 'https://ids.com/.well-known/openid-configuration'.
Microsoft.IdentityModel.Protocols.ConfigurationManager<T>.GetConfigurationAsync(CancellationToken cancel)
Stack Query Cookies Headers 
HttpRequestException: Response status code does not indicate success: 403 (Forbidden).
System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()
Microsoft.IdentityModel.Protocols.HttpDocumentRetriever.GetDocumentAsync(string address, CancellationToken cancel)

    Show raw exception details 
IOException: IDX20804: Unable to retrieve document from: 'https://ids.com/.well-known/openid-configuration'.
Microsoft.IdentityModel.Protocols.HttpDocumentRetriever.GetDocumentAsync(string address, CancellationToken cancel)
Microsoft.IdentityModel.Protocols.OpenIdConnect.OpenIdConnectConfigurationRetriever.GetAsync(string address, IDocumentRetriever retriever, CancellationToken cancel)
Microsoft.IdentityModel.Protocols.ConfigurationManager<T>.GetConfigurationAsync(CancellationToken cancel)

Show raw exception details 
InvalidOperationException: IDX20803: Unable to obtain configuration from: 'https://ids.com/.well-known/openid-configuration'.
Microsoft.IdentityModel.Protocols.ConfigurationManager<T>.GetConfigurationAsync(CancellationToken cancel)
Microsoft.AspNetCore.Authentication.OpenIdConnect.OpenIdConnectHandler.HandleChallengeAsync(AuthenticationProperties properties)
Microsoft.AspNetCore.Authentication.AuthenticationHandler<TOptions>.ChallengeAsync(AuthenticationProperties properties)
Microsoft.AspNetCore.Authentication.AuthenticationService.ChallengeAsync(HttpContext context, string scheme, AuthenticationProperties properties)
Microsoft.AspNetCore.Mvc.ChallengeResult.ExecuteResultAsync(ActionContext context)
Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.InvokeResultAsync(IActionResult result)
Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.InvokeAlwaysRunResultFilters()
Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.InvokeFilterPipelineAsync()
Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.InvokeAsync()
Microsoft.AspNetCore.Builder.RouterMiddleware.Invoke(HttpContext httpContext)
Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context)

Just out of curiosity I shut down my IDS application and tried to access it through one of my other applications and got the exact same error response back. Which leads me to beleive it's something to do with either the code in my applications when using openidc or my IIS settings.

Things to note: I'm using IIS. I have a trusted CA cert on my proxy site. Assuming IDS is running I am able to access 'https://ids.com/.well-known/openid-configuration' in my broswer. You will not be, this is a fake domain name.

things I've tried:

1. inside of OpenIdConnect tried switching RequireHttpsMetadata from false to true,

2. app.UseForwardedHeaders();

Okay I could definitely be wrong here, but what I believe to be the problem is my applications (when using OIDC) are not sending the SSL information which forbids them from access the 'https://ids.com/.well-know' address.

Where should I start with trying to work with that?

Some of my code from startup:

    services.Configure<ForwardedHeadersOptions>(options =>
            {
                options.ForwardedHeaders = ForwardedHeaders.XForwardedFor | ForwardedHeaders.XForwardedProto;
                options.RequireHeaderSymmetry = false;
                options.KnownNetworks.Clear();
                options.KnownProxies.Clear();

            });

    .AddOpenIdConnect(AuthorizationConsts.OidcAuthenticationScheme, options =>
     {

         options.SignInScheme = "Cookies";
         options.Authority = "https://ids.com";
         options.RequireHttpsMetadata = true;
         options.ClientId = AuthorizationConsts.OidcClientId;


         options.Scope.Clear();
         options.Scope.Add(AuthorizationConsts.ScopeOpenId);
         options.Scope.Add(AuthorizationConsts.ScopeProfile);
         options.Scope.Add(AuthorizationConsts.ScopeEmail);
         options.Scope.Add(AuthorizationConsts.ScopeRoles);



         options.SaveTokens = true;

         options.TokenValidationParameters = new TokenValidationParameters
         {
             NameClaimType = JwtClaimTypes.Name,
             RoleClaimType = JwtClaimTypes.Role,
         };

         options.Events = new OpenIdConnectEvents
         {
             OnMessageReceived = OnMessageReceived,
             OnRedirectToIdentityProvider = OnRedirectToIdentityProvider

         };
     });

..............


    app.Use(async (Context, next) =>
           {
               if (!string.IsNullOrEmpty(Context.Request.Headers["X-ARR-SSL"]))
               {
                   Context.Request.Scheme = "https";
                   await next.Invoke();

               }
               else
               {
                   Context.Request.Scheme = "http";
                   await next.Invoke();
               }

           });

            app.UseForwardedHeaders();

            // Ensures we can serve static-files that should not be processed by ASP.NET
            app.UseStaticFiles();

            // Enable the authentication middleware so we can protect access to controllers and/or actions
            app.UseAuthentication();

Answer 1

I finally solved this issue. I knew it had to do with certificates, but I wasn't sure what to do.

My fix was adding options.BackchannelHttpHandler to

private readonly HttpClientHandler _handler;

public Startup(IHostingEnvironment env, IConfiguration config,
    ILoggerFactory loggerFactory)
    {
        _env = env;
        _config = config;
        _loggerFactory = loggerFactory;
        Configuration = config;
        _handler = new HttpClientHandler();

        _handler.ClientCertificates.Add(FindClientCertificate());//same x509cert2 that proxy server uses
        _handler.AllowAutoRedirect = true;




    }
.....


AddOpenIdConnect( scheme, options => {
....
options.BackchannelHttpHandler = _handler;
...
}

Answer 2

For me, the issue got resolved with adding

ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12