ASP.NET Core Web API client does not trust self-signed certificate used by the Identity Server instance

Question

This is a follow up of this question.

I have generated and trusted a self-signed certificate using the following script:

#create a SAN cert for both host.docker.internal and localhost
#$cert = New-SelfSignedCertificate -DnsName "host.docker.internal", "localhost" -CertStoreLocation "cert:\LocalMachine\Root" 
# does not work: New-SelfSignedCertificate : A new certificate can only be installed into MY store.
$cert = New-SelfSignedCertificate -DnsName "host.docker.internal", "localhost" -CertStoreLocation cert:\localmachine\my

#export it for docker container to pick up later
$password = ConvertTo-SecureString -String "password_here" -Force -AsPlainText
Export-PfxCertificate -Cert $cert -FilePath "$env:USERPROFILE\.aspnet\https\aspnetapp.pfx" -Password $password

# trust it on your host machine
$store = New-Object System.Security.Cryptography.X509Certificates.X509Store [System.Security.Cryptography.X509Certificates.StoreName]::Root,"LocalMachine"
$store.Open("ReadWrite")
$store.Add($cert)
$store.Close()

When accessing https://host.docker.internal:5500/.well-known/openid-configuration and https://localhost:5500/.well-known/openid-configuration on the host machine, it works as expected (certificate is OK).

However, the Web API application running in the container is not happy with it:

web_api          | System.InvalidOperationException: IDX20803: Unable to obtain configuration from: 'https://host.docker.internal:5500/.well-known/openid-configuration'.
web_api          |  ---> System.IO.IOException: IDX20804: Unable to retrieve document from: 'https://host.docker.internal:5500/.well-known/openid-configuration'.
web_api          |  ---> System.Net.Http.HttpRequestException: The SSL connection could not be established, see inner exception.
web_api          |  ---> System.Security.Authentication.AuthenticationException: The remote certificate is invalid according to the validation procedure.
web_api          |    at System.Net.Security.SslStream.StartSendAuthResetSignal(ProtocolToken message, AsyncProtocolRequest asyncRequest, ExceptionDispatchInfo exception)

The docker-compose file for the API is the following (relevant parts only):

  web.api:
    image: web_api_image
    build: 
      context: .
      dockerfile: ProjectApi/Dockerfile
    environment: 
      - ASPNETCORE_ENVIRONMENT=ContainerDev 
    container_name: web_api
    ports:
      - "5600:80"
    networks:
      - backend
      - data_layer
    depends_on:
      - identity.server
      - mssqlserver
      - web.cache

  identity.server:
    image: identity_server_image
    build: 
      context: .
      dockerfile: MyProject.IdentityServer/Dockerfile
    environment: 
      - ASPNETCORE_ENVIRONMENT=ContainerDev 
      - ASPNETCORE_URLS=https://+:443;http://+:80
      - ASPNETCORE_Kestrel__Certificates__Default__Password=password_here
      - ASPNETCORE_Kestrel__Certificates__Default__Path=/https/aspnetapp.pfx
    volumes:
      - ~/.aspnet/https:/https:ro
    container_name: identity_server
    ports:
      - "5500:443"
      - "5501:80"
    networks:
      - backend
      - data_layer
    depends_on:
      - mssqlserver

How can I make this work?

---

The call towards the identity server is done by setting up the security in API client to use it (no explicit HTTPS call):

/// <summary>
/// configures authentication and authorization
/// </summary>
/// <param name="services"></param>
/// <param name="configuration"></param>
public static void ConfigureSecurity(this IServiceCollection services, IConfiguration configuration)
{
    string baseUrl = configuration.GetSection("Idam").GetValue<string>("BaseUrl");
    Console.WriteLine($"Authentication server base URL = {baseUrl}");

    services.AddAuthentication(options =>
    {
        options.DefaultAuthenticateScheme = JwtBearerDefaults.AuthenticationScheme;
        options.DefaultChallengeScheme = JwtBearerDefaults.AuthenticationScheme;
    }).AddJwtBearer(o =>
    {
        o.MetadataAddress = $"{baseUrl}/.well-known/openid-configuration";
        o.Authority = "dev_identity_server";
        o.Audience = configuration.GetSection("Idam").GetValue<string>("Audience");
        o.RequireHttpsMetadata = false;
    });

    services.AddAuthorization();
}

---

Identity server configuration

public void ConfigureServices(IServiceCollection services)
{
    string connectionStr = Configuration.GetConnectionString("Default");
    Console.WriteLine($"[Identity server] Connection string = {connectionStr}");

    services.AddDbContext<AppIdentityDbContext>(options => options.UseSqlServer(connectionStr));

    services.AddTransient<AppIdentityDbContextSeedData>();


    services.AddIdentity<AppUser, IdentityRole>()
        .AddEntityFrameworkStores<AppIdentityDbContext>()
        .AddDefaultTokenProviders();

    services.AddIdentityServer(act =>
        {
            act.IssuerUri = "dev_identity_server";
        })
        .AddDeveloperSigningCredential()
        // this adds the operational data from DB (codes, tokens, consents)
        .AddOperationalStore(options =>
        {
            options.ConfigureDbContext = builder => builder.UseSqlServer(Configuration.GetConnectionString("Default"));
            // this enables automatic token cleanup. this is optional.
            options.EnableTokenCleanup = true;
            options.TokenCleanupInterval = 30; // interval in seconds
        })
        //.AddInMemoryPersistedGrants()
        .AddInMemoryIdentityResources(Config.GetIdentityResources())
        .AddInMemoryApiResources(Config.GetApiResources())
        .AddInMemoryClients(Config.GetClients(Configuration))
        .AddAspNetIdentity<AppUser>();

    services.AddDataProtection()
        .PersistKeysToFileSystem(new DirectoryInfo(@"\\UNC-PATH"));

    services.AddTransient<IProfileService, IdentityClaimsProfileService>();

    services.AddCors(options => options.AddPolicy("AllowAll", p => p.AllowAnyOrigin()
       .AllowAnyMethod()
       .AllowAnyHeader()));

    services.AddMvc(options =>
    {
        options.EnableEndpointRouting = false;
    }).SetCompatibilityVersion(CompatibilityVersion.Latest);
}

// This method gets called by the runtime. Use this method to configure the HTTP request pipeline.
public static void Configure(IApplicationBuilder app, IWebHostEnvironment env, 
    ILoggerFactory loggerFactory, AppIdentityDbContextSeedData seeder)
{
    seeder.SeedTestUsers();
    IdentityModelEventSource.ShowPII = true;

    if (env.IsDevelopment())
    {
        app.UseDeveloperExceptionPage();
    }

    app.UseExceptionHandler(builder =>
    {
        builder.Run(async context =>
        {
            context.Response.StatusCode = (int)HttpStatusCode.InternalServerError;
            context.Response.Headers.Add("Access-Control-Allow-Origin", "*");

            var error = context.Features.Get<IExceptionHandlerFeature>();
            if (error != null)
            {
                context.Response.AddApplicationError(error.Error.Message);
                await context.Response.WriteAsync(error.Error.Message).ConfigureAwait(false);
            }
        });
    });

    // app.UseHttpsRedirection();

    app.UseStaticFiles();
    app.UseCors("AllowAll");
    app.UseIdentityServer();

    app.UseMvc(routes =>
    {
        routes.MapRoute(
            name: "default",
            template: "{controller=Home}/{action=Index}/{id?}");
    });
}

Answer 1

After a few attempts I gave up trying to get docker containers to trust a cert generated by New-SelfSignedCertificate (you may try and get it to work - concepts are exactly the same, it's just the certs are somehow different). I did however have success with OpenSSL:

$certPass = "password_here"
$certSubj = "host.docker.internal"
$certAltNames = "DNS:localhost,DNS:host.docker.internal,DNS:identity_server" # i believe you can also add individual IP addresses here like so: IP:127.0.0.1
$opensslPath="path\to\openssl\binaries" #assuming you can download OpenSSL, I believe no installation is necessary
$workDir="path\to\your\project" # i assume this will be your solution root
$dockerDir=Join-Path $workDir "ProjectApi" #you probably want to check if my assumptions about your folder structure are correct

#generate a self-signed cert with multiple domains
Start-Process -NoNewWindow -Wait -FilePath (Join-Path $opensslPath "openssl.exe") -ArgumentList "req -x509 -nodes -days 365 -newkey rsa:2048 -keyout ",
                                          (Join-Path $workDir aspnetapp.key),
                                          "-out", (Join-Path $dockerDir aspnetapp.crt),
                                          "-subj `"/CN=$certSubj`" -addext `"subjectAltName=$certAltNames`""

# this time round we convert PEM format into PKCS#12 (aka PFX) so .net core app picks it up
Start-Process -NoNewWindow -Wait -FilePath (Join-Path $opensslPath "openssl.exe") -ArgumentList "pkcs12 -export -in ", 
                                           (Join-Path $dockerDir aspnetapp.crt),
                                           "-inkey ", (Join-Path $workDir aspnetapp.key),
                                           "-out ", (Join-Path $workDir aspnetapp.pfx),
                                           "-passout pass:$certPass"

$password = ConvertTo-SecureString -String $certPass -Force -AsPlainText
$cert = Get-PfxCertificate -FilePath (Join-Path $workDir "aspnetapp.pfx") -Password $password

# and still, trust it on your host machine
$store = New-Object System.Security.Cryptography.X509Certificates.X509Store [System.Security.Cryptography.X509Certificates.StoreName]::Root,"LocalMachine"
$store.Open("ReadWrite")
$store.Add($cert)
$store.Close()

I used plain Ubuntu image to be able to test this with wget but a quick check indicates that Microsoft images would support the same build steps:

FROM ubuntu:14.04

RUN  apt-get update \
  && apt-get install -y wget \
  && rm -rf /var/lib/apt/lists/*

USER root 
###### you probably only care about the following three lines
ADD ./aspnetapp.crt /usr/local/share/ca-certificates/asp_dev/
RUN chmod -R 644 /usr/local/share/ca-certificates/asp_dev/
RUN update-ca-certificates --fresh
######

ENTRYPOINT tail -f /dev/null

my docker-compose is pretty much identical to yours. I'll list it here for completeness:

version: '3'
services:
  web_api:
    build: ./ProjectApi
    container_name: web_api
    ports:
      - "5600:80"
    depends_on:
      - identity_server

  identity_server:
    image: mcr.microsoft.com/dotnet/core/samples:aspnetapp    
    environment: 
      - ASPNETCORE_URLS=https://+:443;http://+:80
      - ASPNETCORE_Kestrel__Certificates__Default__Password=password_here
      - ASPNETCORE_Kestrel__Certificates__Default__Path=/https/aspnetapp.pfx
    volumes:
      - ~/.aspnet/https/:/https/:ro 
    container_name: identity_server
    ports:
      - "5500:443"
      - "5501:80"

With all of the above, I haven't tested running an actual .net core application as a client on a container - my test was a pretty simple wget https://identity_server.docker.internal command line.

So there's still a chance you might have issues. This would be due to the fact that some applications use their own trusted certs - see this SE thread for more context.

Hopefully though, it's going to be a smooth ride from here.