How to disable constraint check (Netscape cert type) in Java6?

Question

I am trying to build a custom HTTPS Server in Java (6) using the built in class com.sun.net.httpserver.HttpsServer. It works fine until I require client authentication. At that point it fails with the following exception in the SSL debug on the server.

sun.security.validator.ValidatorException: Netscape cert type does not permit use for SSL client

I am using certificates issued by our internal CA which is used for all applications internal to us. I checked the certificate details and found that type was "SSL Server" (details quoted below). Since our policy is to use a "SSL Server" type for all internal applications, it is difficult to change the cerificate. Since I want to use a Server certificate for Client, I don't believe this is a security issue.

What I am looking for is a way disable this constraint check in Java. Has anyone encountered this and solved this? Any help is highly appreciated.

Best Regards, Arun

Owner: CN=myapp, OU=mygroup, O=mycompany

Issuer: O=MYCA

Serial number: 4cc8c1da

Valid from: Mon Jan 10 13:46:34 EST 2011 until: Thu Jan 10 14:16:34 EST 2013

Certificate fingerprints:
         MD5:  8C:84:7F:7A:40:23:F1:B5:81:CD:F9:0C:27:16:69:5E
         SHA1: 9B:39:0B:2F:61:83:52:93:D5:58:E5:43:13:7A:8F:E1:FD:AC:98:A4
         Signature algorithm name: SHA1withRSA
         Version: 3

Extensions:

[1]: ObjectId: 2.5.29.16 Criticality=false
PrivateKeyUsage: [
From: Mon Jan 10 13:46:34 EST 2011, To: Wed Jul 11 21:16:34 EDT 2012]

[2]: ObjectId: 2.5.29.15 Criticality=false
KeyUsage [
  DigitalSignature
  Key_Encipherment
]

[3]: ObjectId: 2.5.29.14 Criticality=false
SubjectKeyIdentifier [
KeyIdentifier [
0000: D3 47 35 9B B4 B7 03 18   C6 53 2C B0 FE FD 49 D8  .G5......S,...I.
0010: D0 FB EE 15                                        ....
]
]

[4]: ObjectId: 1.2.840.113533.7.65.0 Criticality=false

[5]: ObjectId: 2.5.29.31 Criticality=false
CRLDistributionPoints [
  [DistributionPoint:
     [CN=CRL413, O=SWIFT]
]]

[6]: ObjectId: 2.5.29.19 Criticality=false
BasicConstraints:[
  CA:false
  PathLen: undefined
]

****[7]: ObjectId: 2.16.840.1.113730.1.1 Criticality=false
NetscapeCertType [
   SSL server
]****

[8]: ObjectId: 2.5.29.35 Criticality=false
AuthorityKeyIdentifier [
KeyIdentifier [
0000: 8F AF 56 BC 80 77 A3 FD   9E D2 89 83 98 FE 98 C7  ..V..w..........
0010: 20 65 23 CC                                         e#.
]

]

Answer 1

You could wrap the default trust managers and catch this particular exception. This would be something along these lines:

class IgnoreClientUsageTrustManager extends X509TrustManager {
    private final X509TrustManager origTrustManager;
    public class IgnoreClientUsageTrustManager(X509TrustManager origTrustManager) {
        this.origTrustManager = origTrustManager;
    }

    public checkClientTrusted(X509Certificate[] chain, String authType
        throws IllegalArgumentException, CertificateException {
        try {
            this.origTrustManager.checkClientTrusted(chain, authType);
        } catch (ValidatorException e) {
             // Check it's that very exception, otherwise, re-throw.
        }
    }

    // delegate the other methods to the origTrustManager
}        

Then, use that trust manager to create an SSLContext and use it with your server.

TrustManagerFactory tmf = TrustManagerFactory.getInstance(
    TrustManagerFactory.getDefaultAlgorithm());
tmf.init((KeyStore)null);
TrustManager[] trustManagers = tmf.getTrustManagers();

for (int i = 0; i < trustManagers.length; i++) {
    if (trustManagers[i] instanceof X509TrustManager) {
       trustManagers[i] = IgnoreClientUsageTrustManager(trustManagers[i]);
    }
}

SSLContext sslContext = SSLContext.getInstance("TLS");
sslContext.init(... you keymanagers ..., trustManagers, null);

You should initialise your keymanagers from your server keystores (as normal). You should then be able to use the HttpsServer's HttpsConfigurator to set up the SSLContext (see example in the documentation).

This technique isn't ideal, though.

• Firstly, ValidatorException is in a sun.* package that's not part of the public API: this code will be specific for the Oracle/OpenJDK JRE.
• Secondly, it relies on the fact that the end entity checked (which verifies the key usage extension) happens after the rest of the trust validation (which makes it acceptable to ignore that exception, since you don't ignore other more fundamental checks this way).

You could of course re-implement your own validation instead, using the Java Certificate Path API, and ignoring only the key usage for this purpose. This requires a bit more code.

More generally, you're trying to bypass the specifications anyway, if you want to use a certificate for SSL/TLS as a client certificate when it doesn't have the right extension. The best fix for this is to amend your CA policy, which should be feasible if it's an internal CA anyway. It's quite common for server certificates to have the TLS client extended key usage set too, even with big CAs.