Javamail NTLM Authentication Failure

Question

Trying to connect to Exchange server using NTLM in JavaMail. I can connect to SMTP, but not IMAP. I can also authenticate via the OS X Mail.app application using the identical host/username/password, account type = "IMAP", Port 143, ssl=false, authentication=NTLM, Domain Name="".

The connecting code:

import javax.mail.Session;
import javax.mail.Transport;
import javax.mail.Store;
import java.util.Properties;

    public class NTLMTest {
        public static void main(String[] args) throws Exception {
            final String host = "example.com";
            final String user = "bob";
            final String password = "password";

            final Properties properties = new Properties();
            Session session = Session.getDefaultInstance(properties);
            session.setDebug(true);

            // SMTP CONNECT
            final Transport transport = session.getTransport("smtp");
            transport.connect(host, user, password);
            System.out.println("SMTP Connect successful");

            // IMAP CONNECT
            final Store store = session.getStore("imap");
            store.connect(host, user, password);
            System.out.println("IMAP Connect Successful");

        }
    }

The output:

DEBUG: setDebug: JavaMail version 1.4.3
DEBUG: getProvider() returning javax.mail.Provider[TRANSPORT,smtp,com.sun.mail.smtp.SMTPTransport,Sun Microsystems, Inc]
DEBUG SMTP: useEhlo true, useAuth false
DEBUG SMTP: trying to connect to host "example.com", port 25, isSSL false
220 server18.example.com ESMTP Sendmail 8.14.3/8.14.3/Debian-5+lenny1; Thu, 2 Dec 2010 18:05:30 +0100; (No UCE/UBE) logging access from: xxx.xxx.xxx.xxx
DEBUG SMTP: connected to host "example.com", port: 25

EHLO 192.168.1.107
250-server18.example.com Hello c-xxxx [xxx.xxx.xxx.xxx], pleased to meet you
250-ENHANCEDSTATUSCODES
250-PIPELINING
250-8BITMIME
250-SIZE 20971520
250-DSN
250-ETRN
250-AUTH DIGEST-MD5 CRAM-MD5 LOGIN PLAIN
250-STARTTLS
250-DELIVERBY
250 HELP
DEBUG SMTP: Found extension "ENHANCEDSTATUSCODES", arg ""
DEBUG SMTP: Found extension "PIPELINING", arg ""
DEBUG SMTP: Found extension "8BITMIME", arg ""
DEBUG SMTP: Found extension "SIZE", arg "20971520"
DEBUG SMTP: Found extension "DSN", arg ""
DEBUG SMTP: Found extension "ETRN", arg ""
DEBUG SMTP: Found extension "AUTH", arg "DIGEST-MD5 CRAM-MD5 LOGIN PLAIN"
DEBUG SMTP: Found extension "STARTTLS", arg ""
DEBUG SMTP: Found extension "DELIVERBY", arg ""
DEBUG SMTP: Found extension "HELP", arg ""
DEBUG SMTP: Attempt to authenticate
DEBUG SMTP: check mechanisms: LOGIN PLAIN DIGEST-MD5 NTLM 
AUTH LOGIN
334 VXNlcm5hbWU6
YWR2aWVzZW5raWVzMDU=
334 UGFzc3dvcmQ6
ZGlja2hvbmluZw==
235 2.0.0 OK Authenticated
SMTP Connect successful
DEBUG: getProvider() returning javax.mail.Provider[STORE,imap,com.sun.mail.imap.IMAPStore,Sun Microsystems, Inc]
DEBUG: mail.imap.fetchsize: 16384
DEBUG: mail.imap.statuscachetimeout: 1000
DEBUG: mail.imap.appendbuffersize: -1
DEBUG: mail.imap.minidletime: 10
DEBUG: trying to connect to host "example.com", port 143, isSSL false
* OK server18.example.com Cyrus IMAP4 v2.1.18-IPv6-Debian-2.1.18-5.1 server ready
A0 CAPABILITY
* CAPABILITY IMAP4 IMAP4rev1 ACL QUOTA LITERAL+ MAILBOX-REFERRALS NAMESPACE UIDPLUS ID NO_ATOMIC_RENAME UNSELECT CHILDREN MULTIAPPEND SORT THREAD=ORDEREDSUBJECT THREAD=REFERENCES IDLE AUTH=DIGEST-MD5 AUTH=NTLM AUTH=CRAM-MD5 ANNOTATEMORE
A0 OK Completed
IMAP DEBUG: AUTH: DIGEST-MD5
IMAP DEBUG: AUTH: NTLM
IMAP DEBUG: AUTH: CRAM-MD5
DEBUG: protocolConnect login, host=example.com, user=bob, password=<non-null>
DEBUG NTLM: type 1 message: Type1Message[suppliedDomain=,suppliedWorkstation=192.168.1.107,flags=0x00000201]
DEBUG NTLM: type 1 message length: 45
A1 AUTHENTICATE NTLM
+ 
TlRMTVNTUAABAAAAASIAAAAAAAAAAAAADQANACAAAAAxOTIuMTY4LjEuMTA3
+ TlRMTVNTUAACAAAAAAAAADAAAAABIgAApdhJrA6NzmwAAAAAAAAAAAAAAAAAAAAA
TlRMTVNTUAADAAAAGAAYAEAAAAAwADAAWAAAAAAAAAAAAAAAHAAcAIgAAAAaABoApAAAAAAAAAAAAAAAAQIAALV6mIutJKdZSH4IZGmvNqNFxJafzInd0yJDR4J3oe3LyBls0Y75UuwBAQAAAAAAANAS9yNDkssBVbH5v087iUIAAAAAAAAAAGEAZAB2AGkAZQBzAGUAbgBrAGkAZQBzADAANQAxADkAMgAuADEANgA4AC4AMQAuADEAMAA3AA==
A1 NO authentication failure
Exception in thread "main" javax.mail.AuthenticationFailedException: authentication failure
    at com.sun.mail.imap.IMAPStore.protocolConnect(IMAPStore.java:613)
    at javax.mail.Service.connect(Service.java:291)
    at javax.mail.Service.connect(Service.java:172)
    at com.prosc.emailplugin.NTLMTest.main(NTLMTest.java:25)
Disconnected from the target VM, address: '127.0.0.1:56125', transport: 'socket'

Process finished with exit code 1

I tried wrapping the username with backslashes, per http://www.oracle.com/technetwork/java/faq-135477.html#Exchange-login I get the following error:

Exception in thread "main" javax.mail.AuthenticationFailedException: One time use of a plaintext password will enable requested mechanism for user

Backslashes around the username in the SMTP connect portion cause it to fail. I can't tell if the "One time use" error is a step in the right direction or not.

Answer 1

I have noticed that - via SMTP - the NTLM Authentication did not work with an older version of javax.mail (up to 1.4.1) but it now works with version 1.4.5. And the user name to be specified was of the form "domain\username". Could be that same effect (difference in versions of javax.mail) would also apply to IMAP.

Answer 2

From what I recall about NTLM, and your NTLM debug messages I can gather the following:

• NTLM was designed for single sign on, thus taking credentials from the windows machine its running on - especially the JDK implementation of NTLM.
  • NTLM has two versions, well three to be exact. NTLM v1, NTLMv2 and another version I can't recall at the moment. NTLM v1 has a security hole which allows you to really use a username and password and connect using the NTLM protocol. In NTLM v2 it was fixed, which forces the implementation to take the password (the hashed pass) from the logged in Windows machine.
  • It seems that the NTLM protocol in your case stops after the first message sent by the Exchange server. Notice it has flags which declares which type of NTLM is used and other such as : Encryption, etc.

I suggest you try following the road in which the credentials (u/p) are taken automatically by the JDK from the client Windows machine.

Answer 3

Here is my complete working solution:

Using netbeans, create a new java application project. Put this code in there:

package javaapplication4;
import java.util.Date;
import java.util.Properties;

import javax.mail.Message;
import javax.mail.PasswordAuthentication;
import javax.mail.Session;
import javax.mail.Transport;
import javax.mail.internet.MimeMessage;


public class JavaApplication4 {
    public static void main(String[] args) throws Exception {
           new JavaApplication4().send_email();
    }
    private void send_email() throws Exception
    {
        Properties props = new Properties();
        props.put("mail.smtp.host", "smtp.yourserver.net");
        props.put("mail.from", "[email protected]");
        props.put("mail.smtp.starttls.enable", "true");
        props.put("mail.smtp.ssl.enable", "false");
        props.put("mail.smtp.auth", "true");
        props.put("mail.smtp.port", "587");

        Authenticator authenticator = new Authenticator();
        props.setProperty("mail.smtp.submitter", authenticator.getPasswordAuthentication().getUserName());

        Session session = Session.getInstance(props, authenticator);
        MimeMessage msg = new MimeMessage(session);
        msg.setFrom();
        msg.setRecipients(Message.RecipientType.TO, "[email protected]");  
            // also tried @gmail.com
        msg.setSubject("JavaMail ssl test");
        msg.setSentDate(new Date());
        msg.setText("Hello, world!\n");

        Transport transport;
        transport = session.getTransport("smtp");
        transport.connect();
        msg.saveChanges(); 
        transport.sendMessage(msg, msg.getAllRecipients());
        transport.close();
    }
    private class Authenticator extends javax.mail.Authenticator {
       private PasswordAuthentication authentication;
       public Authenticator() {
           String username = "[email protected]";
           String password = "yourpassword";
           authentication = new PasswordAuthentication(username, password);
       }
       protected PasswordAuthentication getPasswordAuthentication() {
           return authentication;
       }
   }
}

Change the username, passwords, ports, and properties to your specific settings.

You will need to get the javamail-1.4.7 and load the mail.jar from (http://www.oracle.com/technetwork/java/index-138643.html) into the project.

One thing we did which shed light on what our parameters should be is to download Thunderbird mail client which can auto-discover information about the exchange server to make sure all of our settings were right. If you cant convince thunderbird to connect, then this code likewise shouldn't be expected to work.

Answer 4

I had the same problem sending email via Exhange SMTP connector. After finding out that javamail does not support NTLMv2 authentication, I worked out a workaround that requires JCIFS library though.

I downloded the javamail api source code (https://java.net/projects/javamail/pages/Home) and edited the class com.sun.mail.auth.Ntlm to add the missing support for NTLMv2 using the JCIFS library support (http://jcifs.samba.org).

The first modification in file Ntlm.java was in method init0, and consisted of adding the missing flag NTLMSSP_NEGOTIATE_NTLM2:

private void init0() {
// ANDREA LUCIANO:
//    turn on the NTLMv2 negotiation flag in TYPE1 messages: 
//    NTLMSSP_NEGOTIATE_NTLM2   (0x00080000) 
//  See also http://davenport.sourceforge.net/ntlm.html#type1MessageExample

    type1 = new byte[256];
    type3 = new byte[256];
    System.arraycopy(new byte[] {'N','T','L','M','S','S','P',0,1}, 0,
            type1, 0, 9);
    type1[12] = (byte) 3;
    type1[13] = (byte) 0xb2;
    type1[14] = (byte) 0x08;  // ANDREA LUCIANO - added
// ...

The second modification was to replace the method generateType3Msg with this:

public String generateType3Msg(String challenge) {
    /* First decode the type2 message */
    byte[] type2 = null;
    try {
       type2 = BASE64DecoderStream.decode(challenge.getBytes("us-ascii"));
       logger.fine("type 2 message: " + toHex(type2)); // ANDREA LUCIANO - added
    } catch (UnsupportedEncodingException ex) {
       // should never happen
       assert false;
    }
    jcifs.ntlmssp.Type2Message t2m;
    try {
          t2m = new jcifs.ntlmssp.Type2Message(type2);
    } catch (IOException ex) {
          logger.log(Level.FINE, "Invalid Type 2 message", ex);
          return "";   // will fail later
    }

    final int type2Flags = t2m.getFlags();
    final int type3Flags = type2Flags & (0xffffffff ^ (jcifs.ntlmssp.NtlmFlags.NTLMSSP_TARGET_TYPE_DOMAIN | jcifs.ntlmssp.NtlmFlags.NTLMSSP_TARGET_TYPE_SERVER));

    jcifs.ntlmssp.Type3Message t3m = new jcifs.ntlmssp.Type3Message(t2m, password, ntdomain, username, hostname, type3Flags);
       return jcifs.util.Base64.encode(t3m.toByteArray());
}

The simpest way I found to patch the library is to compile the class and update the library jar file:

"c:\Program Files\Java\jdk1.5.0_22\bin\javac.exe" -cp jcifs-1.3.17.jar;javax.mail-1.5.2.jar -d . Ntlm.java 
 jar uvf javax.mail-1.5.2.jar com\sun\mail\auth\Ntlm.class

To enable debug as much as possible, I used the following code in the main method of my test class:

    final InputStream inputStream = Main.class.getResourceAsStream("/logging.properties");
    LogManager.getLogManager().readConfiguration(inputStream);

    Properties props = new Properties();

    props.put("mail.debug", "true");
    props.put("mail.debug.auth", "true");

with this logging.properties:

   # Logging
   handlers = java.util.logging.ConsoleHandler

  .level = INFO

  # Console Logging
  java.util.logging.ConsoleHandler.level = INFO

Before applying the patch the test was stuck after sending Type 1 message, because my Exchange server required NTLMv2 authentication. After the patch the athentication was done successfully.

Another solution is to send email via the ##Exchange webservice## aka EWS by using the ##EWS Java API## released and mantained by Microsoft (https://github.com/OfficeDev/ews-java-api), such as in this example:

public class Main {

/**
 * @param args
 */
public static void main(String[] args) throws Exception {

       ExchangeService exchangeService = new ExchangeService(ExchangeVersion.Exchange2007_SP1);

        ExchangeCredentials credentials = new WebCredentials("myusername","mypassword", "mydomain");

        exchangeService.setCredentials(credentials);
        exchangeService.setUrl(new URI("https://myhostname/EWS/Exchange.asmx"));

        exchangeService.setTraceEnabled(true);

        EmailMessage msg;
        msg = new EmailMessage(exchangeService);
        msg.setFrom(new EmailAddress("[email protected]"));
        msg.setSubject("Test Mail");
        msg.getToRecipients().add("[email protected]");
        msg.setBody(MessageBody.getMessageBodyFromText("Email sent by Miscrosoft Java EWS API."));
        msg.getAttachments().addFileAttachment("c:\\temp\\myattachement.pdf");
        msg.send();

}

}

But again there is a patch to apply in the inner class NTLM of EwsJCIFSNTLMScheme.java to enable NTLMv2 as indicated in the answer to this post:

How to use LDAP authentication for the Exchange Web Services connection in Java?

That is:

private class NTLM {
/** Character encoding */
public static final String DEFAULT_CHARSET = "ASCII";

/**
* The character was used by 3.x's NTLM to encode the username and
* password. Apparently, this is not needed in when passing username,
* password from NTCredentials to the JCIFS library
*/
private String credentialCharset = DEFAULT_CHARSET;

void setCredentialCharset(String credentialCharset) {
       this.credentialCharset = credentialCharset;
}

private static final int TYPE_1_FLAGS = NtlmFlags.NTLMSSP_NEGOTIATE_NTLM
             | NtlmFlags.NTLMSSP_NEGOTIATE_UNICODE
             | NtlmFlags.NTLMSSP_NEGOTIATE_NTLM2;

private String generateType1Msg(String host, String domain) {
       jcifs.ntlmssp.Type1Message t1m = new jcifs.ntlmssp.Type1Message(
                    TYPE_1_FLAGS, domain, host);
       return jcifs.util.Base64.encode(t1m.toByteArray());
}

private String generateType3Msg(String username, String password,
             String host, String domain, String challenge) {
       jcifs.ntlmssp.Type2Message t2m;
       try {
             t2m = new jcifs.ntlmssp.Type2Message(
                           jcifs.util.Base64.decode(challenge));
       } catch (IOException e) {
             throw new RuntimeException("Invalid Type2 message", e);
       }

       final int type2Flags = t2m.getFlags();
       final int type3Flags = type2Flags
                    & (0xffffffff ^ (NtlmFlags.NTLMSSP_TARGET_TYPE_DOMAIN | NtlmFlags.NTLMSSP_TARGET_TYPE_SERVER));

       jcifs.ntlmssp.Type3Message t3m = new jcifs.ntlmssp.Type3Message(
                    t2m, password, domain, username, host, type3Flags);
       return jcifs.util.Base64.encode(t3m.toByteArray());
}

}

I tried and it worked for me.