HttpClient check Kerberos secured webpage. NTLM login didn't work

Question

I have to write a program which checks a Kerberos secured site of our company. I try it with HttpClient and get following error:

KrbException: Server not found in Kerberos database (7)
    at sun.security.krb5.KrbTgsRep.<init>(KrbTgsRep.java:61)
    at sun.security.krb5.KrbTgsReq.getReply(KrbTgsReq.java:185)
        ...

I have written a NTLM login 5 months ago, but it doesn't work for this Kerberos secured site. I think Nego2 is activated, so it doesn't fall back to NTLM if Kerberos fails.

I read the Kerberos Wikipedia article: http://en.wikipedia.org/wiki/Kerberos_(protocol) and I think the problem is that the TGS doesn't find the requested service in the database. I think this because I get a ticket before the error (I think its the TGT).

Found ticket for [email protected] to go to krbtgt/[email protected] expiring on Thu May 31 01:35:56 CEST 2012

So the error whould happen at "Client Service Authorization" --> 2. (see Wikipedia article)

The problem is that the service has to be in the kerberos database, because I can access it with the IE (without login, so single sign on works).

So my question is: Why does the TGS not find the server in the Kerberos database, but it works with IE?

Extra Information
I tried to get the necessary information on the top but here are all the information because I don't know if I got all necessary information:

OS is Windows 7
Firefox Version 9.0.1
Chrome Version 19.0.1084.52
Safari Version 5.0.2
IE Version 8.0.7600.16385

My Java Code:

    System.setProperty("java.security.auth.login.config", "file://c:/temp/jaas.conf");
    System.setProperty("java.security.krb5.conf", "c:/winnt/krb5.ini");
    System.setProperty("sun.security.krb5.debug", "true");
    System.setProperty("javax.security.auth.useSubjectCredsOnly","false");

    DefaultHttpClient httpclient = new DefaultHttpClient();
    try {
        httpclient.getAuthSchemes().register(AuthPolicy.SPNEGO, new SPNegoSchemeFactory());

        Credentials use_jaas_creds = new Credentials() {

            public String getPassword() {
                return null;
            }

            public Principal getUserPrincipal() {
                return null;
            }

        };

        httpclient.getCredentialsProvider().setCredentials(
                new AuthScope(null, -1, null),
                use_jaas_creds);

        HttpUriRequest request = new HttpGet("url.com:port/site"); //Kerberos secured url
        HttpResponse response = httpclient.execute(request);
        HttpEntity entity = response.getEntity();

        System.out.println("----------------------------------------");
        System.out.println(response.getStatusLine());
        System.out.println("----------------------------------------");
        if (entity != null) {
            System.out.println(EntityUtils.toString(entity));
        }
        System.out.println("----------------------------------------");

        // This ensures the connection gets released back to the manager
        EntityUtils.consume(entity);

jaas.conf

com.sun.security.jgss.login {
com.sun.security.auth.module.Krb5LoginModule required client=TRUE;
};
com.sun.security.jgss.initiate {
com.sun.security.auth.module.Krb5LoginModule required client=TRUE;
};
com.sun.security.jgss.accept {
com.sun.security.auth.module.Krb5LoginModule required client=TRUE;
};

krb5.ini

[logging]
 default = FILE:log/krb5libs.log
 kdc = FILE:log/krb5kdc.log
 admin_server = FILE:log/kadmind.log

[libdefaults]
 ticket_lifetime = 24000
 default_realm = EXAMPLE.COM
 dns_lookup_realm = false
 dns_lookup_kdc = false

[realms]
 EXAMPLE.COM = {
  kdc = url
  admin_server = url
 }

[domain_realm]
 url.com = EXAMPLE.COM

[kdc]
 profile = /var/kerberos/krb5kdc/kdc.conf

[appdefaults]
 pam = {
   debug = false
   ticket_lifetime = 36000
   renew_lifetime = 36000
   forwardable = true
   krb4_convert = false
 }

Hole Log file from a test run:

log4j:WARN No appenders could be found for logger (org.apache.http.impl.conn.BasicClientConnectionManager).
log4j:WARN Please initialize the log4j system properly.
Kerberos-Benutzername [user]: user
Kerberos-Passwort für user: *******
Using builtin default etypes for default_tkt_enctypes
default etypes for default_tkt_enctypes: 3 1 23 16 17.
Using builtin default etypes for default_tkt_enctypes
default etypes for default_tkt_enctypes: 3 1 23 16 17.
>>> KrbAsReq calling createMessage
>>> KrbAsReq in createMessage
>>> KrbKdcReq send: kdc=kdcurl UDP:88, timeout=30000, number of retries =3, #bytes=155
>>> KDCCommunication: kdc=kdcurl UDP:88, timeout=30000,Attempt =1, #bytes=155
>>> KrbKdcReq send: #bytes read=220
>>> KrbKdcReq send: #bytes read=220
>>> KDCRep: init() encoding tag is 126 req type is 11
>>>KRBError:
     sTime is Thu May 31 08:46:29 CEST 2012 1338446789000
     suSec is 51983
     error code is 25
     error Message is Additional pre-authentication required
     realm is EXAMPLE.COM
     sname is krbtgt/EXAMPLE.COM
     eData provided.
     msgType is 30
>>>Pre-Authentication Data:
     PA-DATA type = 11
     PA-ETYPE-INFO etype = 23
>>>Pre-Authentication Data:
     PA-DATA type = 2
     PA-ENC-TIMESTAMP
>>>Pre-Authentication Data:
     PA-DATA type = 15
AcquireTGT: PREAUTH FAILED/REQUIRED, re-send AS-REQ
Using builtin default etypes for default_tkt_enctypes
default etypes for default_tkt_enctypes: 3 1 23 16 17.
Pre-Authentication: Set preferred etype = 23
>>>KrbAsReq salt is EXAMPLE.COMuser
Pre-Authenticaton: find key for etype = 23
AS-REQ: Add PA_ENC_TIMESTAMP now
>>> EType: sun.security.krb5.internal.crypto.ArcFourHmacEType
>>> KrbAsReq calling createMessage
>>> KrbAsReq in createMessage
>>> KrbKdcReq send: kdc=kdcurl UDP:88, timeout=30000, number of retries =3, #bytes=219
>>> KDCCommunication: kdc=kdcurl UDP:88, timeout=30000,Attempt =1, #bytes=219
>>> KrbKdcReq send: #bytes read=100
>>> KrbKdcReq send: #bytes read=100
>>> KDCRep: init() encoding tag is 126 req type is 11
>>>KRBError:
     sTime is Thu May 31 08:46:29 CEST 2012 1338446789000
     suSec is 114485
     error code is 52
     error Message is Response too big for UDP, retry with TCP
     realm is EXAMPLE.COM
     sname is krbtgt/EXAMPLE.COM
     msgType is 30
>>> KrbKdcReq send: kdc=kdcurl TCP:88, timeout=30000, number of retries =3, #bytes=219
>>>DEBUG: TCPClient reading 3277 bytes
>>> KrbKdcReq send: #bytes read=3277
>>> KrbKdcReq send: #bytes read=3277
>>> EType: sun.security.krb5.internal.crypto.ArcFourHmacEType
>>> KrbAsRep cons in KrbAsReq.getReply user
Using builtin default etypes for default_tkt_enctypes
default etypes for default_tkt_enctypes: 3 1 23 16 17.
Found ticket for [email protected] to go to krbtgt/[email protected] expiring on Thu May 31 18:46:29 CEST 2012
Entered Krb5Context.initSecContext with state=STATE_NEW
Service ticket not found in the subject
>>> Credentials acquireServiceCreds: same realm
Using builtin default etypes for default_tgs_enctypes
default etypes for default_tgs_enctypes: 3 1 23 16 17.
>>> CksumType: sun.security.krb5.internal.crypto.RsaMd5CksumType
>>> EType: sun.security.krb5.internal.crypto.ArcFourHmacEType
>>> KrbKdcReq send: kdc=kdcurl UDP:88, timeout=30000, number of retries =3, #bytes=3298
>>> KDCCommunication: kdc=kdcurl UDP:88, timeout=30000,Attempt =1, #bytes=3298
>>> KrbKdcReq send: #bytes read=110
>>> KrbKdcReq send: #bytes read=110
>>> KDCRep: init() encoding tag is 126 req type is 13
>>>KRBError:
     sTime is Thu May 31 08:46:29 CEST 2012 1338446789000
     suSec is 192613
     error code is 7
     error Message is Server not found in Kerberos database
     realm is EXAMPLE.COM
     sname is HTTP/url.com:port
     msgType is 30
KrbException: Server not found in Kerberos database (7)
    at sun.security.krb5.KrbTgsRep.<init>(KrbTgsRep.java:61)
    at sun.security.krb5.KrbTgsReq.getReply(KrbTgsReq.java:185)
    at sun.security.krb5.internal.CredentialsUtil.serviceCreds(CredentialsUtil.java:294)
    at sun.security.krb5.internal.CredentialsUtil.acquireServiceCreds(CredentialsUtil.java:106)
    at sun.security.krb5.Credentials.acquireServiceCreds(Credentials.java:562)
    at sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:594)
    at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:230)
    at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:162)
    at sun.security.jgss.spnego.SpNegoContext.GSS_initSecContext(SpNegoContext.java:851)
    at sun.security.jgss.spnego.SpNegoContext.initSecContext(SpNegoContext.java:309)
    at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:230)
    at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:162)
    at org.apache.http.impl.auth.GGSSchemeBase.generateGSSToken(GGSSchemeBase.java:99)
    at org.apache.http.impl.auth.SPNegoScheme.generateToken(SPNegoScheme.java:80)
    at org.apache.http.impl.auth.GGSSchemeBase.authenticate(GGSSchemeBase.java:155)
    at org.apache.http.impl.auth.SPNegoScheme.authenticate(SPNegoScheme.java:75)
    at org.apache.http.client.protocol.RequestAuthenticationBase.authenticate(RequestAuthenticationBase.java:125)
    at org.apache.http.client.protocol.RequestAuthenticationBase.process(RequestAuthenticationBase.java:83)
    at org.apache.http.client.protocol.RequestTargetAuthentication.process(RequestTargetAuthentication.java:80)
    at org.apache.http.protocol.ImmutableHttpProcessor.process(ImmutableHttpProcessor.java:109)
    at org.apache.http.protocol.HttpRequestExecutor.preProcess(HttpRequestExecutor.java:176)
    at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:516)
    at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
    at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
    at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
    at mypackage.ClientKerberosAuthentication.main(ClientKerberosAuthentication.java:152)
Caused by: KrbException: Identifier doesn't match expected value (906)
    at sun.security.krb5.internal.KDCRep.init(KDCRep.java:133)
    at sun.security.krb5.internal.TGSRep.init(TGSRep.java:58)
    at sun.security.krb5.internal.TGSRep.<init>(TGSRep.java:53)
    at sun.security.krb5.KrbTgsRep.<init>(KrbTgsRep.java:46)
    ... 25 more
----------------------------------------
HTTP/1.1 401 Unauthorized
----------------------------------------
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Draft//EN">
<HTML>
<HEAD>
<TITLE>Error 401--Unauthorized</TITLE>
</HEAD>
<BODY bgcolor="white">
<FONT FACE=Helvetica><BR CLEAR=all>
<TABLE border=0 cellspacing=5><TR><TD><BR CLEAR=all>
<FONT FACE="Helvetica" COLOR="black" SIZE="3"><H2>Error 401--Unauthorized</H2>
</FONT></TD></TR>
</TABLE>
<TABLE border=0 width=100% cellpadding=10><TR><TD VALIGN=top WIDTH=100% BGCOLOR=white><FONT FACE="Courier New"><FONT FACE="Helvetica" SIZE="3"><H3>From RFC 2068 <i>Hypertext Transfer Protocol -- HTTP/1.1</i>:</H3>
</FONT><FONT FACE="Helvetica" SIZE="3"><H4>10.4.2 401 Unauthorized</H4>
</FONT><P><FONT FACE="Courier New">The request requires user authentication. The response MUST include a WWW-Authenticate header field (section 14.46) containing a challenge applicable to the requested resource. The client MAY repeat the request with a suitable Authorization header field (section 14.8). If the request already included Authorization credentials, then the 401 response indicates that authorization has been refused for those credentials. If the 401 response contains the same challenge as the prior response, and the user agent has already attempted authentication at least once, then the user SHOULD be presented the entity that was given in the response, since that entity MAY include relevant diagnostic information. HTTP access authentication is explained in section 11.</FONT></P>
</FONT></TD></TR>
</TABLE>

</BODY>
</HTML>

----------------------------------------

Answer 1

It won't work.

1. Active Directory does not use any ports in the SPNs. Don't know where that stupid stuff came into the HttpClient.
2. There is no SPN registered for your target host in your Active Directory.

Answers to your questions:

1. Not at all, only service class and hostname are mandatory. The service class already identifies the service with which the port is registered, e.g. LDAP, HTTP, FTP and so forth. We have hundreds of thousands SPNs in our forest. None of them with ports. I would a tremendous work to register every single port instance of a HTTP server, for example. The AD has to know the target host only. It will encrypt the service ticket accordingly. This makes it unique.
2. Why should the browser fail? It does construct a SPN as HTTP/<FQDN>. As long as this one exists in the directory, everything will go smoothly.

When I tried the HTTPClient, I was never happy with the crappy SPNEGO support. I'd like to rewrite that stuff when I get the hands on.

Answer 2

i found the way to solve this problem.
Just add a parameter(true) to the constructor of SPNegoSchemeFactory object.

  httpclient.getAuthSchemes().register(AuthPolicy.SPNEGO, new SPNegoSchemeFactory(true));

This will make your sname HTTP/url.com:port to HTTP/url.com.
Thanks to this jira: https://issues.apache.org/jira/browse/HTTPCLIENT-966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Answer 3

I created a small tool to simplify connecting with httpclient 3 to kerberos (no external configuration files), you might want to give it a try. https://github.com/DovAmir/httpclientAuthHelper

DefaultHttpClient httpclient = new DefaultHttpClient();
AuthUtils.securityLogging(SecurityLogType.KERBEROS,true);
CredentialsUtils.setKerberosCredentials(client, new UsernamePasswordCredentials("xxx", "xxx"), "domain", "kdc");
client.executeMethod(httpget);