Office 365 XOAUTH2 for IMAP and SMTP Authentication fails

Question

Recently the support for OAuth 2.0 for IMAP and SMTP in the Exchange Online has been announced. Following the guide I've set up the application permissions and IMAP and SMTP connection. The application is configured as Accounts in any organizational directory (Any Azure AD directory - Multitenant) and uses authorization code flow.

URLs below are used for authorization:

• https://login.microsoftonline.com/organizations/oauth2/v2.0/authorize
• https://login.microsoftonline.com/organizations/oauth2/v2.0/token

And the following Delegated Microsoft Graph scopes have been added:

The scopes, requests from code:

final List<String> scopes = Arrays.asList(
    "offline_access",
    "email",
    "openid",
    "profile",
    "User.Read",
    "Mail.ReadWrite",
    "https%3A%2F%2Foutlook.office365.com%2FIMAP.AccessAsUser.All",
    "https%3A%2F%2Foutlook.office365.com%2FSMTP.Send"
);

I successfully receive the access and refresh tokens:

{
    "token_type": "Bearer",
    "scope": "email IMAP.AccessAsUser.All Mail.ReadWrite openid profile SMTP.Send User.Read",
    "expires_in": 3599,
    "ext_expires_in": 3599,
    "access_token": "edited",
    "refresh_token": "edited",
    "id_token": "edited"
}

Here's the code, used to connect to IMAP:

Properties props = new Properties();
props.put("mail.imap.ssl.enable", "true");
props.put("mail.imaps.sasl.enable", "true");
props.put("mail.imaps.sasl.mechanisms", "XOAUTH2");
props.put("mail.imap.auth.login.disable", "true");
props.put("mail.imap.auth.plain.disable", "true");
props.put("mail.debug", "true");
props.put("mail.debug.auth", "true");

Session session = Session.getInstance(props);
session.setDebug(true);

String userEmail = "[email protected]";
String accessToken = "access_token_received_on_previous_step";

final Store store = session.getStore("imaps");
store.connect("outlook.office365.com", 993, userEmail, accessToken);

Which generates the following output:

DEBUG: JavaMail version 1.6.2
DEBUG: successfully loaded resource: /META-INF/javamail.default.address.map
DEBUG: setDebug: JavaMail version 1.6.2
DEBUG: getProvider() returning javax.mail.Provider[STORE,imaps,com.sun.mail.imap.IMAPSSLStore,Oracle]
DEBUG IMAPS: mail.imap.fetchsize: 16384
DEBUG IMAPS: mail.imap.ignorebodystructuresize: false
DEBUG IMAPS: mail.imap.statuscachetimeout: 1000
DEBUG IMAPS: mail.imap.appendbuffersize: -1
DEBUG IMAPS: mail.imap.minidletime: 10
DEBUG IMAPS: enable SASL
DEBUG IMAPS: SASL mechanisms allowed: XOAUTH2
DEBUG IMAPS: closeFoldersOnStoreFailure
DEBUG IMAPS: trying to connect to host "outlook.office365.com", port 993, isSSL true
* OK The Microsoft Exchange IMAP4 service is ready. [QQBNADc...]
A0 CAPABILITY
* CAPABILITY IMAP4 IMAP4rev1 AUTH=PLAIN AUTH=XOAUTH2 SASL-IR UIDPLUS MOVE ID UNSELECT CHILDREN IDLE NAMESPACE LITERAL+
A0 OK CAPABILITY completed.
DEBUG IMAPS: AUTH: PLAIN
DEBUG IMAPS: AUTH: XOAUTH2
DEBUG IMAPS: protocolConnect login, host=outlook.office365.com, [email protected], password=<non-null>
DEBUG IMAPS: SASL Mechanisms:
DEBUG IMAPS:  XOAUTH2
DEBUG IMAPS: 
DEBUG IMAPS: SASL client XOAUTH2
DEBUG IMAPS: SASL callback length: 2
DEBUG IMAPS: SASL callback 0: javax.security.auth.callback.NameCallback@17046283
DEBUG IMAPS: SASL callback 1: javax.security.auth.callback.PasswordCallback@5bd03f44
A1 AUTHENTICATE XOAUTH2 dXNlcj1o...
A1 NO AUTHENTICATE failed.

Exception in thread "main" javax.mail.AuthenticationFailedException: AUTHENTICATE failed.
    at com.sun.mail.imap.IMAPStore.protocolConnect(IMAPStore.java:732)
    at javax.mail.Service.connect(Service.java:366)

And the following code is used for connecting to SMTP:

Properties props = new Properties();
props.put("mail.smtp.auth", "true");
props.put("mail.transport.protocol", "smtp");
props.put("mail.smtp.starttls.enable", "true");
props.put("mail.smtp.auth.mechanisms", "XOAUTH2");
props.put("mail.smtp.auth.login.disable","true");
props.put("mail.smtp.auth.plain.disable","true");
props.put("mail.debug.auth", "true");

Session session = Session.getInstance(props);
session.setDebug(true);

String userEmail = "[email protected]";
String accessToken = "access_token_received_on_previous_step";

Transport transport = session.getTransport("smtp");
transport.connect("smtp.office365.com", 587, userEmail, accessToken);

Which provides the output below:

DEBUG: setDebug: JavaMail version 1.6.2
DEBUG: getProvider() returning javax.mail.Provider[TRANSPORT,smtp,com.sun.mail.smtp.SMTPTransport,Oracle]
DEBUG SMTP: useEhlo true, useAuth true
DEBUG SMTP: trying to connect to host "smtp.office365.com", port 587, isSSL false
220 AM5PR0701CA0005.outlook.office365.com Microsoft ESMTP MAIL Service ready at Mon, 4 May 2020 15:52:28 +0000
DEBUG SMTP: connected to host "smtp.office365.com", port: 587
EHLO ubuntu-B450-AORUS-M
250-AM5PR0701CA0005.outlook.office365.com Hello [my ip here]
250-SIZE 157286400
250-PIPELINING
250-DSN
250-ENHANCEDSTATUSCODES
250-STARTTLS
250-8BITMIME
250-BINARYMIME
250-CHUNKING
250 SMTPUTF8
DEBUG SMTP: Found extension "SIZE", arg "157286400"
DEBUG SMTP: Found extension "PIPELINING", arg ""
DEBUG SMTP: Found extension "DSN", arg ""
DEBUG SMTP: Found extension "ENHANCEDSTATUSCODES", arg ""
DEBUG SMTP: Found extension "STARTTLS", arg ""
DEBUG SMTP: Found extension "8BITMIME", arg ""
DEBUG SMTP: Found extension "BINARYMIME", arg ""
DEBUG SMTP: Found extension "CHUNKING", arg ""
DEBUG SMTP: Found extension "SMTPUTF8", arg ""
STARTTLS
220 2.0.0 SMTP server ready
EHLO ubuntu-B450-AORUS-M
250-AM5PR0701CA0005.outlook.office365.com Hello [my ip here]
250-SIZE 157286400
250-PIPELINING
250-DSN
250-ENHANCEDSTATUSCODES
250-AUTH LOGIN XOAUTH2
250-8BITMIME
250-BINARYMIME
250-CHUNKING
250 SMTPUTF8
DEBUG SMTP: Found extension "SIZE", arg "157286400"
DEBUG SMTP: Found extension "PIPELINING", arg ""
DEBUG SMTP: Found extension "DSN", arg ""
DEBUG SMTP: Found extension "ENHANCEDSTATUSCODES", arg ""
DEBUG SMTP: Found extension "AUTH", arg "LOGIN XOAUTH2"
DEBUG SMTP: Found extension "8BITMIME", arg ""
DEBUG SMTP: Found extension "BINARYMIME", arg ""
DEBUG SMTP: Found extension "CHUNKING", arg ""
DEBUG SMTP: Found extension "SMTPUTF8", arg ""
DEBUG SMTP: protocolConnect login, host=smtp.office365.com, [email protected], password=<non-null>
DEBUG SMTP: Attempt to authenticate using mechanisms: XOAUTH2
DEBUG SMTP: Using mechanism XOAUTH2
AUTH XOAUTH2 dXNlcj1obW9kaUB...
535 5.7.3 Authentication unsuccessful [AM5PR0701CA0005.eurprd07.prod.outlook.com]

Exception in thread "main" javax.mail.AuthenticationFailedException: 535 5.7.3 Authentication unsuccessful [AM5PR0701CA0005.eurprd07.prod.outlook.com]
    at com.sun.mail.smtp.SMTPTransport$Authenticator.authenticate(SMTPTransport.java:965)
    at com.sun.mail.smtp.SMTPTransport.authenticate(SMTPTransport.java:876)
    at com.sun.mail.smtp.SMTPTransport.protocolConnect(SMTPTransport.java:780)
    at javax.mail.Service.connect(Service.java:366)

What I've also tried:

• specifying scopes as https://graph.microsoft.com/SMTP.Send and just SMTP.Send
• using https://login.microsoftonline.com/common/ url for authentication

Result is always the same.

Is it something I do wrong or there's a bug somewhere in the support for this from the Microsoft side?

Update 1:

Tried from the command line, but same result:

$ openssl s_client -crlf -connect outlook.office365.com:993
... connection part omitted
* OK The Microsoft Exchange IMAP4 service is ready. [QQBNADYAUAAxADkAMgBDAEEAMAAwADkAMQAuAEUAVQBSAFAAMQA5ADIALgBQAFIATwBEAC4ATwBVAFQATABPAE8ASwAuAEMATwBNAA==]
C01 CAPABILITY
* CAPABILITY IMAP4 IMAP4rev1 AUTH=PLAIN AUTH=XOAUTH2 SASL-IR UIDPLUS ID UNSELECT CHILDREN IDLE NAMESPACE LITERAL+
C01 OK CAPABILITY completed.
A01 AUTHENTICATE XOAUTH2 dXNlcj1obW9kaUBjb...
A01 NO AUTHENTICATE failed.
* BYE Connection is closed. 13
read:errno=0

Update 2:

Tried to create brand new application in the Azure Portal with the following permissions:

And receiving the following screen, when trying to give consent for scopes:

That is odd, because the permissions from Azure Portal don't specify that the Admin consent is required and my previous app registration doesn't show such screen when IMAP and SMTP scopes are requested.

Update 3:

Thanks to comments to this post I tried the following scopes:

public static final List<String> SCOPES = Arrays.asList(
    "offline_access",
    "https%3A%2F%2Foutlook.office365.com%2FIMAP.AccessAsUser.All",
    "https%3A%2F%2Foutlook.office365.com%2FSMTP.Send"
);

Which gave me the token below:

{
    "token_type": "Bearer",
    "scope": "https://outlook.office365.com/IMAP.AccessAsUser.All https://outlook.office365.com/SMTP.Send",
    "expires_in": 3599,
    "ext_expires_in": 3599,
    "access_token": "eyJ0eXAiOiJKV1....",
    "refresh_token": "OAQABAAAAAAAm...."
}

IMAP/SMTP auth was successful and I was able to read the inbox + send an email!

But for my application I need also couple of other scopes to use some MS Graph API endpoints (read user profile, messages subscription and messages deletion).

So I tried different scopes:

public static final List<String> SCOPES = Arrays.asList(
        "offline_access",
        "User.Read",
        "Mail.ReadWrite",
        "https%3A%2F%2Foutlook.office365.com%2FIMAP.AccessAsUser.All",
        "https%3A%2F%2Foutlook.office365.com%2FSMTP.Send"
);

This gave the token (note that scope value differs from the token that actually worked, the permissions don't have outlook url):

{
    "token_type": "Bearer",
    "scope": "IMAP.AccessAsUser.All Mail.ReadWrite SMTP.Send User.Read profile openid email",
    "expires_in": 3599,
    "ext_expires_in": 3599,
    "access_token": "eyJ0eXAiOiJKV1Q...",
    "refresh_token": "OAQABAAAAAAAm..."
}

Which led to the result I got previously:

A1 NO AUTHENTICATE failed.

Trying all the scopes to be as URLs:

public static final List<String> SCOPES = Arrays.asList(
        "offline_access", // or "https%3A%2F%2Fgraph.microsoft.com%2Foffline_access"
        "https%3A%2F%2Fgraph.microsoft.com%2FUser.Read",
        "https%3A%2F%2Fgraph.microsoft.com%2FMail.ReadWrite",
        "https%3A%2F%2Foutlook.office365.com%2FIMAP.AccessAsUser.All",
        "https%3A%2F%2Foutlook.office365.com%2FSMTP.Send"
);

Leads to the following error when obtaining the token (the consent step passed successfully):

{
    "error": "invalid_request",
    "error_description": "AADSTS28000: Provided value for the input parameter scope is not valid because it contains more than one resource. Scope offline_access https://graph.microsoft.com/user.read https://graph.microsoft.com/mail.readwrite https://outlook.office365.com/imap.accessasuser.all https://outlook.office365.com/smtp.send is not valid.\r\nTrace ID: c3282396-6231-4e11-8300-77bc2ca57f00\r\nCorrelation ID: 5f5145bf-7114-4e6c-ab11-30e7ff84a056\r\nTimestamp: 2020-05-06 08:08:48Z",
    "error_codes": [
        28000
    ],
    "timestamp": "2020-05-06 08:08:48Z",
    "trace_id": "c3282396-6231-4e11-8300-77bc2ca57f00",
    "correlation_id": "5f5145bf-7114-4e6c-ab11-30e7ff84a056"
}

And when trying all the scopes to have microsoft graph (as copied from the Azure Portal)

public static final List<String> SCOPES = Arrays.asList(
    "https%3A%2F%2Fgraph.microsoft.com%2Foffline_access",
    "https%3A%2F%2Fgraph.microsoft.com%2FUser.Read",
    "https%3A%2F%2Fgraph.microsoft.com%2FMail.ReadWrite",
    "https%3A%2F%2Fgraph.microsoft.com%2FIMAP.AccessAsUser.All",
    "https%3A%2F%2Fgraph.microsoft.com%2FSMTP.Send"
);

Return the following token (without a refresh token althout offline_access has been requested)

{
    "token_type": "Bearer",
    "scope": "profile openid email https://graph.microsoft.com/IMAP.AccessAsUser.All https://graph.microsoft.com/Mail.ReadWrite https://graph.microsoft.com/SMTP.Send https://graph.microsoft.com/User.Read",
    "expires_in": 3599,
    "ext_expires_in": 3599,
    "access_token": "eyJ0eXAiOiJKV1..."
}

No success:

A1 NO AUTHENTICATE failed.

So it appears that if you don't specify Outlook url for scope it's assumed probably as Graph one which doesn't allow authorization through IMAP and SMTP.

Update 4:

By requesting all the scopes I need at consent step, then getting first access token with only Graph scopes and the second one using refresh token endpoint specifying Outlook scopes - it worked. Refresh token method for getting second access token is used because if you try to obtains access token by auth code you'll get get the following error:

{
    "error": "invalid_grant",
    "error_description": "AADSTS54005: OAuth2 Authorization code was already redeemed, please retry with a new valid code or use an existing refresh token.\r\nTrace ID: 09fc80f4-f5fd-4e52-938f-d56b71dd0900\r\nCorrelation ID: 4f35e05c-23c8-4fdc-a5a7-2fcde5a73b44\r\nTimestamp: 2020-05-08 12:13:30Z",
    "error_codes": [
        54005
    ],
    "timestamp": "2020-05-08 12:13:30Z",
    "trace_id": "09fc80f4-f5fd-4e52-938f-d56b71dd0900",
    "correlation_id": "4f35e05c-23c8-4fdc-a5a7-2fcde5a73b44"
}

So no I'll need to use two separate tokens depending on what resource I'll need to manage.

Update 5:

If it still doesn't work - check if your organization has Security Default enabled - they disable POP/IMAP/SMTP auth for accounts - https://techcommunity.microsoft.com/t5/exchange-team-blog/announcing-oauth-2-0-support-for-imap-and-smtp-auth-protocols-in/bc-p/1544725/highlight/true#M28589

Answer 1

IMAP, SMTP scopes are targeted for Exchange resource and not Graph. Whereas User.Read, Mail.ReadWrite are meant for Graph resource.

We do not support generation of tokens that are meant for two resources. Hence the error "Provided value for the input parameter scope is not valid because it contains more than one resource."

You should generate two tokens separately by two calls to /token. 1. One with the IMAP, SMTP scopes generated for the Exchange resource. 2. The other with Graph scopes (User.Read, Mail.ReadWrite) meant for Graph resource.