"Defective token detected" error (NTLM not Kerberos) with Kerberos/Spring Security/IE/Active Directory

Question

We are having trouble getting Spring Security/Kerberos/AD to work for our web app. Our diagnosis is that our AD server sending an NTLM token (we can tell as it starts with "TlRMTVNT.....") to IE and IE is then sending this to our application and it's failing. Our AD server should be sending a Kerberos/SPNEGO token to IE.

The "moving parts" are as follows:

• Spring Security 3.0 (patched)
• Microsoft Windows Server Enterprise 2003 SP1 Active Directory
• IE 8
• Tomcat (TC Server 6.0)
• Java 1.6

We have set everything up as detailed in the instructions here:

https://spring.io/blog/2009/09/28/spring-security-kerberos-spnego-extension

This involved:

• Creating a normal user as the Service Principal (the same as the machine name where our application resides). We set the following account options:
  • disabled 'Use has to change password at next logon'
  • enabled 'password never expires'
  • enabled 'Use Kerberos DES…'
  • disabled 'Do not require Kerberos preauthentication'
  • NOTE: Server 2003 does not present the 'This account supports Kerberos AES 128 bit…' and 'This account supports Kerberos AES 256 bit…' options
• Used "ktpass.exe" to assign the service principal name (SPN) to this new user and export this user key to a keytab file. using 'ktpass /out ourweb.keytab /mapuser [email protected] /princ HTTP/[email protected] /pass *'
• Downloaded source code from https://src.springframework.org/svn/se-security/trunk.
• Copied the keytab file from the AD server to WEB-INF/etc of source code (application).
• Made change to file SunJaasKerbersoTicketValidator.java to read keytab file. (To resolve the bug in which the application can't read the keytab file from the Java classpath) options.put("keyTab", "C:\se-security\spring-security-kerberos\spring-security-kerberos-sample\src\main\webapp\WEB-INF\etc\ourweb.keytab");
• Configured web.xml to use spnego.xml. contextConfigLocation /WEB-INF/spnego.xml
  • Configured Spring Security (spnego.xml) to use Kerberos (SpnegoEntryPoint, SpnegoAuthenticationProcessingFilter and KerberosServiceAuthenticationProvider beans) by providing our service prinicipal name and keytab file location.
  • Configured spnego.xml to read keytab file copied in WEB-INF/etc.

When we started up our TC server we could see things initializing nicely (i.e. without error - "principles key obtained from the keytab"):

Creating instance of bean 'org.springframework.security.extensions.kerberos.SunJaasKerberosTicketValidator#10fa4b8'  Invoking afterPropertiesSet() on bean with name 'org.springframework.security.extensions.kerberos.SunJaasKerberosTicketValidator#10fa4b8'  Config name: C:\WINDOWS\krb5.ini Debug is  true storeKey true useTicketCache false useKeyTab true doNotPrompt true ticketCache is null isInitiator false KeyTab is C:\se-security\spring-security-kerberos\spring-security-kerberos-sample\src\main\webapp\WEB-INF\etc\ourwebapp4.keytab refreshKrb5Config is false principal is HTTP/ourwebappweb4.testdomain.ourcompany.co.uk tryFirstPass is false useFirstPass is false storePass is false clearPass is false >>> KeyTabInputStream, readName(): TESTDOMAIN.OURCOMPANY.CO.UK >>> KeyTabInputStream, readName(): HTTP >>> KeyTabInputStream, readName(): ourweb >>> KeyTab: load() entry length: 78; type: 1 >>> KeyTabInputStream, readName(): TESTDOMAIN.OURCOMPANY.CO.UK >>> KeyTabInputStream, readName(): HTTP >>> KeyTabInputStream, readName(): ourweb.testdomain.ourcompany.co.uk >>> KeyTab: load() entry length: 113; type: 1 Added key: 1version: 2 Ordering keys wrt default_tkt_enctypes list default etypes for default_tkt_enctypes: 1. 0: EncryptionKey: keyType=1 kvno=2 keyValue (hex dump)= 0000: 91 01 43 E3 02 A8 B9 83     principal's key obtained from the keytab principal is HTTP/[email protected] EncryptionKey: keyType=1 keyBytes (hex dump)=0000: 91 01 43 E3 02 A8 B9 83    Added server's keyKerberos Principal HTTP/ourweb.testdomain.ourcompany.co.uk@TESTDOMAIN.OURCOMPANY.CO.UKKey Version 2key EncryptionKey: keyType=1 keyBytes (hex dump)= 0000: 91 01 43 E3 02 A8 B9 83     [Krb5LoginModule] added Krb5Principal  HTTP/[email protected] to Subject Commit Succeeded   Finished creating instance of bean 'org.springframework.security.extensions.kerberos.SunJaasKerberosTicketValidator#10fa4b8'  

Ready to test, we then enabled "Windows Integrated Authentication" in IE and made sure that the domain was listed in IE's local intranet site section. We then connected to our web application using the fully qualified domain name.

When we did so we got the following errors in the browser:

500 Internal server error. 

and in the TC Server log file:

Negotiate Header was invalid: Negotiate     TlRMTVNTUAABAAAAB4IIogAAAAAAAAAAAAAAAAAAAAAFASgKAAAADw==    org.springframework.security.authentication.BadCredentialsException: Kerberos validation not succesfull   at org.springframework.security.extensions.kerberos.SunJaasKerberosTicketValidator.validateTicket(SunJaasKerberosTicketValidator.java:74)   at org.springframework.security.extensions.kerberos.KerberosServiceAuthenticationProvider.authenticate(KerberosServiceAuthenticationProvider.java:92)   at org.springframework.security.authentication.ProviderManager.doAuthentication(ProviderManager.java:120)   at org.springframework.security.authentication.AbstractAuthenticationManager.authenticate(AbstractAuthenticationManager.java:48)   at org.springframework.security.extensions.kerberos.web.SpnegoAuthenticationProcessingFilter.doFilter(SpnegoAuthenticationProcessingFilter.java:132)   at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:355)   at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:79)   at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:355)   at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:149)   at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:237)   at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:167)   at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)   at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)   at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)   at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)   at com.springsource.metrics.collection.web.HttpRequestMetricCollectionValve.invoke(HttpRequestMetricCollectionValve.java:44)   at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)   at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)   at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)   at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)   at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:849)   at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583)   at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:379)   at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)   at java.lang.Thread.run(Thread.java:619) Caused by: java.security.PrivilegedActionException: GSSException: Defective token detected (Mechanism level: GSSHeader did not find the right tag)   at java.security.AccessController.doPrivileged(Native Method)   at javax.security.auth.Subject.doAs(Subject.java:396)   at org.springframework.security.extensions.kerberos.SunJaasKerberosTicketValidator.validateTicket(SunJaasKerberosTicketValidator.java:72)   ... 25 more Caused by: GSSException: Defective token detected (Mechanism level: GSSHeader did not find the right tag)   at sun.security.jgss.GSSHeader.<init>(GSSHeader.java:80)   at sun.security.jgss.GSSContextImpl.acceptSecContext(GSSContextImpl.java:287)   at sun.security.jgss.GSSContextImpl.acceptSecContext(GSSContextImpl.java:267)   at org.springframework.security.extensions.kerberos.SunJaasKerberosTicketValidator$KerberosValidateAction.run(SunJaasKerberosTicketValidator.java:161)   at org.springframework.security.extensions.kerberos.SunJaasKerberosTicketValidator$KerberosValidateAction.run(SunJaasKerberosTicketValidator.java:1)   ... 28 more SecurityContextHolder now cleared, as request processing completed 

It seems (from what we can make out) that the AD server sending an NTLM token (we can tell as it starts with "TlRMTVNT.....") to IE and IE is then sending this to our application and it's failing.

Our AD server should be sending a Kerberos/SPNEGO token to IE.

Other notes:

• Our server(tc server) and client(browser) are on different (virtual) machines and in same domain.

Answer 1

This can happen when you are running the client and server on the same machine. When you use IE to talk to the machine running tomcat ensure that these are distinct machines.

Additionally you need to ensure that the server machine is joined to the domain specified in the keytab (testdomain.ourcompany.co.uk) or you might drop back to NTLM. Your keytab can still work even if your server is on a machine not joined to the domain (you'll see the nice keytab decrypt that you showed), but IE can get confused and not do the correct thing.

AD only really likes to speak arcfour-hmac for Server 2003 so you need to ensure that you set this up correctly in your krb5.ini file.

You can correctly create the keytab like this:

C:\>ktpass -princ HTTP/[email protected] -mapuser [email protected] -crypto RC4-HMAC-NT -ptype K RB5_NT_PRINCIPAL -pass * -out ourweb.keytab Targeting domain controller: test-dc.ourcompany.co.uk Using legacy password setting method Successfully mapped HTTP/[email protected] to ourweb.testdomain.ourcompany.co.uk. Key created. Output keytab to ourweb.keytab: Keytab version: 0x502 keysize 75 HTTP/[email protected] ptype 1 (KRB5_NT_PRINCIPAL) vno 3 etype 0x17 (RC4-HMAC) keylength 16 (0x0fd0e500225c4fca9a63a9998b17ca32) 

I did not see that you had set up a krb5.ini file. You will need to have that set correctly on your server machine (default location C:\WINDOWS\krb5.ini):

[domain_realm]       .testdomain.ourcompany.co.uk = TESTDOMAIN.OURCOMPANY.CO.UK     testdomain.ourcompany.co.uk = TESTDOMAIN.OURCOMPANY.CO.UK  [libdefaults]        default_realm = TESTDOMAIN.OURCOMPANY.CO.UK     permitted_enctypes = aes128-cts aes256-cts arcfour-hmac-md5      default_tgs_enctypes = aes128-cts aes256-cts arcfour-hmac-md5      default_tkt_enctypes = aes128-cts aes256-cts arcfour-hmac-md5   [realms]     VERDAD.LOCAL = {             kdc = test-dc.ourcompany.co.uk       admin_server = test-dc.ourcompany.co.uk     default_domain = TESTDOMAIN.OURCOMPANY.CO.UK } 

You might also need to set the following properties (if you are trying to run this from an IDE):

<systemProperties>   <java.security.krb5.kdc>test-dc.ourcompany.co.uk</java.security.krb5.kdc>   <java.security.krb5.realm>TESTDOMAIN.OURCOMPANY.CO.UK</java.security.krb5.realm> </systemProperties> 

I was using the org.codehaus.mojo plugin for maven which sets these in the pom file like this:

<build>   <plugins>     <plugin>       <groupId>org.codehaus.mojo</groupId>       <artifactId>tomcat-maven-plugin</artifactId>       <configuration>         <server>tomcat-development-server</server>         <port>8080</port>         <path>/SecurityTest</path>         <systemProperties>           <java.security.krb5.kdc>test-dc.ourcompany.co.uk</java.security.krb5.kdc           <java.security.krb5.realm>TESTDOMAIN.OURCOMPANY.CO.UK</java.security.krb5.realm>         </systemProperties>       </configuration>     </plugin>   </plugins> </build> 

Answer 2

I also encountered this problem. For those unlucky people who will have this problem in the future, another cause for this problem is accessing the server by ip instead of it's A record (hostname)