How to use a client certificate to authenticate and authorize in a Web API

Question

I am trying to use a client certificate to authenticate and authorize devices using a Web API and developed a simple proof of concept to work through issues with the potential solution. I am running into an issue where the client certificate is not being received by the web application. A number of people of reported this issue, including in this Q&A, but none of them have an answer. My hope is to provide more detail to revive this issue and hopefully get an answer for my issue. I am open to other solutions. The main requirement is that a standalone process written in C# can call a Web API and be authenticated using a client certificate.

The Web API in this POC is very simple and just returns a single value. It uses an attribute to validate that HTTPS is used and that a client certificate is present.

public class SecureController : ApiController
{
    [RequireHttps]
    public string Get(int id)
    {
        return "value";
    }

}

Here is the code for the RequireHttpsAttribute:

public class RequireHttpsAttribute : AuthorizationFilterAttribute 
{ 
    public override void OnAuthorization(HttpActionContext actionContext) 
    { 
        if (actionContext.Request.RequestUri.Scheme != Uri.UriSchemeHttps) 
        { 
            actionContext.Response = new HttpResponseMessage(System.Net.HttpStatusCode.Forbidden) 
            { 
                ReasonPhrase = "HTTPS Required" 
            }; 
        } 
        else 
        {
            var cert = actionContext.Request.GetClientCertificate();
            if (cert == null)
            {
                actionContext.Response = new HttpResponseMessage(System.Net.HttpStatusCode.Forbidden)
                {
                    ReasonPhrase = "Client Certificate Required"
                }; 

            }
            base.OnAuthorization(actionContext); 
        } 
    } 
}

In this POC I am just checking for the availability of the client certificate. Once this is working I can add checks for information in the certificate to validate against a list of certificates.

Here are the setting in IIS for SSL for this web application.

Here is the code for the client that sends the request with a client certificate. This is a console application.

    private static async Task SendRequestUsingHttpClient()
    {
        WebRequestHandler handler = new WebRequestHandler();
        X509Certificate certificate = GetCert("ClientCertificate.cer");
        handler.ClientCertificates.Add(certificate);
        handler.ServerCertificateValidationCallback = new RemoteCertificateValidationCallback(ValidateServerCertificate);
        handler.ClientCertificateOptions = ClientCertificateOption.Manual;
        using (var client = new HttpClient(handler))
        {
            client.BaseAddress = new Uri("https://localhost:44398/");
            client.DefaultRequestHeaders.Accept.Clear();
            client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));

            HttpResponseMessage response = await client.GetAsync("api/Secure/1");
            if (response.IsSuccessStatusCode)
            {
                string content = await response.Content.ReadAsStringAsync();
                Console.WriteLine("Received response: {0}",content);
            }
            else
            {
                Console.WriteLine("Error, received status code {0}: {1}", response.StatusCode, response.ReasonPhrase);
            }
        }
    }

    public static bool ValidateServerCertificate(
      object sender,
      X509Certificate certificate,
      X509Chain chain,
      SslPolicyErrors sslPolicyErrors)
    {
        Console.WriteLine("Validating certificate {0}", certificate.Issuer);
        if (sslPolicyErrors == SslPolicyErrors.None)
            return true;

        Console.WriteLine("Certificate error: {0}", sslPolicyErrors);

        // Do not allow this client to communicate with unauthenticated servers.
        return false;
    }

When I run this test app I get back a status code of 403 Forbidden with a reason phrase of “Client Certificate Required” indicating that it is getting into my RequireHttpsAttribute and it is not finding any client certificates. Running this through a debugger I have verified that the certificate is getting loaded and added to the WebRequestHandler. The certificate is exported into a CER file that is being loaded. The full certificate with the private key is located on the Local Machine’s Personal and Trusted Root stores for the web application server. For this test the client and web application are being run on the same machine.

I can call this Web API method using Fiddler, attaching the same client certificate, and it works fine. When using Fiddler it passes the tests in RequireHttpsAttribute and returns a successful status code of 200 and returns the expected value.

Has anybody run into the same issue where HttpClient does not send a client certificate in the request and found a solution?

Update 1:

I also tried getting the certificate from the certificate store that includes the private key. Here is how I retrieved it:

    private static X509Certificate2 GetCert2(string hostname)
    {
        X509Store myX509Store = new X509Store(StoreName.My, StoreLocation.LocalMachine);
        myX509Store.Open(OpenFlags.ReadWrite);
        X509Certificate2 myCertificate = myX509Store.Certificates.OfType<X509Certificate2>().FirstOrDefault(cert => cert.GetNameInfo(X509NameType.SimpleName, false) == hostname);
        return myCertificate;
    }

I verified that this certificate was getting retrieved correctly and it was being added to the client certificate collection. But I got the same results where the server code does not retrieve any client certificates.

For completeness here is the code used to retrieve the certificate from a file:

    private static X509Certificate GetCert(string filename)
    {
        X509Certificate Cert = X509Certificate.CreateFromCertFile(filename);
        return Cert;

    }

You will notice that when you get the certificate from a file it returns an object of type X509Certificate and when you retrieve it from the certificate store it is of type X509Certificate2. The X509CertificateCollection.Add Method is expecting a type of X509Certificate.

Update 2: I am still trying to figure this out and have tried many different options but to no avail.

• I changed the web application to run on a host name instead of local host.
• I set the web application to require SSL
• I verified that the certificate was set for Client Authentication and that it is in the trusted root
• Besides testing the client certificate in Fiddler I also validated it in Chrome.

At one point during trying these options it started working. Then I started backing out changes to see what caused it to work. It continued to work. Then I tried removing the certificate from the trusted root to validate that this was required and it stopped working and now I cannot get it back to working even though I put the certificate back in the trusted root. Now Chrome will not even prompt me for a certificate like it used too and it fails in Chrome, but still works in Fiddler. There must be some magic configuration I am missing.

I also tried enabling "Negotiate Client Certificate" in the binding but Chrome still will not prompt me for a client certificate. Here is the settings using "netsh http show sslcert"

 IP:port                 : 0.0.0.0:44398
 Certificate Hash        : 429e090db21e14344aa5d75d25074712f120f65f
 Application ID          : {4dc3e181-e14b-4a21-b022-59fc669b0914}
 Certificate Store Name  : MY
 Verify Client Certificate Revocation    : Disabled
 Verify Revocation Using Cached Client Certificate Only    : Disabled
 Usage Check    : Enabled
 Revocation Freshness Time : 0
 URL Retrieval Timeout   : 0
 Ctl Identifier          : (null)
 Ctl Store Name          : (null)
 DS Mapper Usage    : Disabled
 Negotiate Client Certificate    : Enabled

Here is the client certificate I am using:

I am baffled as to what the issue is. I am adding a bounty for anyone that can help me figure this out.

Answer 1

Tracing helped me find what the problem was (Thank you Fabian for that suggestion). I found with further testing that I could get the client certificate to work on another server (Windows Server 2012). I was testing this on my development machine (Window 7) so I could debug this process. So by comparing the trace to an IIS Server that worked and one that did not I was able to pinpoint the relevant lines in the trace log. Here is a portion of a log where the client certificate worked. This is the setup right before the send

System.Net Information: 0 : [17444] InitializeSecurityContext(In-Buffers count=2, Out-Buffer length=0, returned code=CredentialsNeeded).
System.Net Information: 0 : [17444] SecureChannel#54718731 - We have user-provided certificates. The server has not specified any issuers, so try all the certificates.
System.Net Information: 0 : [17444] SecureChannel#54718731 - Selected certificate:

Here is what the trace log looked like on the machine where the client certificate failed.

System.Net Information: 0 : [19616] InitializeSecurityContext(In-Buffers count=2, Out-Buffer length=0, returned code=CredentialsNeeded).
System.Net Information: 0 : [19616] SecureChannel#54718731 - We have user-provided certificates. The server has specified 137 issuer(s). Looking for certificates that match any of the issuers.
System.Net Information: 0 : [19616] SecureChannel#54718731 - Left with 0 client certificates to choose from.
System.Net Information: 0 : [19616] Using the cached credential handle.

Focusing on the line that indicated the server specified 137 issuers I found this Q&A that seemed similar to my issue. The solution for me was not the one marked as an answer since my certificate was in the trusted root. The answer is the one under it where you update the registry. I just added the value to the registry key.

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL

Value name: SendTrustedIssuerList Value type: REG_DWORD Value data: 0 (False)

After adding this value to the registry it started to work on my Windows 7 machine. This appears to be a Windows 7 issue.

Answer 2

Update:

Example from Microsoft:

https://learn.microsoft.com/en-us/azure/app-service/app-service-web-configure-tls-mutual-auth#special-considerations-for-certificate-validation

Original

This is how I got client certification working and checking that a specific Root CA had issued it as well as it being a specific certificate.

First I edited <src>\.vs\config\applicationhost.config and made this change: <section name="access" overrideModeDefault="Allow" />

This allows me to edit <system.webServer> in web.config and add the following lines which will require a client certification in IIS Express. Note: I edited this for development purposes, do not allow overrides in production.

For production follow a guide like this to set up the IIS:

https://medium.com/@hafizmohammedg/configuring-client-certificates-on-iis-95aef4174ddb

web.config:

<security>
  <access sslFlags="Ssl,SslNegotiateCert,SslRequireCert" />
</security>

API Controller:

[RequireSpecificCert]
public class ValuesController : ApiController
{
    // GET api/values
    public IHttpActionResult Get()
    {
        return Ok("It works!");
    }
}

Attribute:

public class RequireSpecificCertAttribute : AuthorizationFilterAttribute
{
    public override void OnAuthorization(HttpActionContext actionContext)
    {
        if (actionContext.Request.RequestUri.Scheme != Uri.UriSchemeHttps)
        {
            actionContext.Response = new HttpResponseMessage(System.Net.HttpStatusCode.Forbidden)
            {
                ReasonPhrase = "HTTPS Required"
            };
        }
        else
        {
            X509Certificate2 cert = actionContext.Request.GetClientCertificate();
            if (cert == null)
            {
                actionContext.Response = new HttpResponseMessage(System.Net.HttpStatusCode.Forbidden)
                {
                    ReasonPhrase = "Client Certificate Required"
                };

            }
            else
            {
                X509Chain chain = new X509Chain();

                //Needed because the error "The revocation function was unable to check revocation for the certificate" happened to me otherwise
                chain.ChainPolicy = new X509ChainPolicy()
                {
                    RevocationMode = X509RevocationMode.NoCheck,
                };
                try
                {
                    var chainBuilt = chain.Build(cert);
                    Debug.WriteLine(string.Format("Chain building status: {0}", chainBuilt));

                    var validCert = CheckCertificate(chain, cert);

                    if (chainBuilt == false || validCert == false)
                    {
                        actionContext.Response = new HttpResponseMessage(System.Net.HttpStatusCode.Forbidden)
                        {
                            ReasonPhrase = "Client Certificate not valid"
                        };
                        foreach (X509ChainStatus chainStatus in chain.ChainStatus)
                        {
                            Debug.WriteLine(string.Format("Chain error: {0} {1}", chainStatus.Status, chainStatus.StatusInformation));
                        }
                    }
                }
                catch (Exception ex)
                {
                    Debug.WriteLine(ex.ToString());
                }
            }

            base.OnAuthorization(actionContext);
        }
    }

    private bool CheckCertificate(X509Chain chain, X509Certificate2 cert)
    {
        var rootThumbprint = WebConfigurationManager.AppSettings["rootThumbprint"].ToUpper().Replace(" ", string.Empty);

        var clientThumbprint = WebConfigurationManager.AppSettings["clientThumbprint"].ToUpper().Replace(" ", string.Empty);

        //Check that the certificate have been issued by a specific Root Certificate
        var validRoot = chain.ChainElements.Cast<X509ChainElement>().Any(x => x.Certificate.Thumbprint.Equals(rootThumbprint, StringComparison.InvariantCultureIgnoreCase));

        //Check that the certificate thumbprint matches our expected thumbprint
        var validCert = cert.Thumbprint.Equals(clientThumbprint, StringComparison.InvariantCultureIgnoreCase);

        return validRoot && validCert;
    }
}

Can then call the API with client certification like this, tested from another web project.

[RoutePrefix("api/certificatetest")]
public class CertificateTestController : ApiController
{

    public IHttpActionResult Get()
    {
        var handler = new WebRequestHandler();
        handler.ClientCertificateOptions = ClientCertificateOption.Manual;
        handler.ClientCertificates.Add(GetClientCert());
        handler.UseProxy = false;
        var client = new HttpClient(handler);
        var result = client.GetAsync("https://localhost:44331/api/values").GetAwaiter().GetResult();
        var resultString = result.Content.ReadAsStringAsync().GetAwaiter().GetResult();
        return Ok(resultString);
    }

    private static X509Certificate GetClientCert()
    {
        X509Store store = null;
        try
        {
            store = new X509Store(StoreName.My, StoreLocation.CurrentUser);
            store.Open(OpenFlags.OpenExistingOnly | OpenFlags.ReadOnly);

            var certificateSerialNumber= "‎81 c6 62 0a 73 c7 b1 aa 41 06 a3 ce 62 83 ae 25".ToUpper().Replace(" ", string.Empty);

            //Does not work for some reason, could be culture related
            //var certs = store.Certificates.Find(X509FindType.FindBySerialNumber, certificateSerialNumber, true);

            //if (certs.Count == 1)
            //{
            //    var cert = certs[0];
            //    return cert;
            //}

            var cert = store.Certificates.Cast<X509Certificate>().FirstOrDefault(x => x.GetSerialNumberString().Equals(certificateSerialNumber, StringComparison.InvariantCultureIgnoreCase));

            return cert;
        }
        finally
        {
            store?.Close();
        }
    }
}

Answer 3

Make sure HttpClient has access to the full client certificate (including the private key).

You are calling GetCert with a file "ClientCertificate.cer" which leads to the assumption that there is no private key contained - should rather be a pfx file within windows. It may be even better to access the certificate from the windows cert store and search it using the fingerprint.

Be careful when copying the fingerprint: There are some non-printable characters when viewing in cert management (copy the string over to notepad++ and check the length of the displayed string).