NodeMailer - send mail with Google service account fails because "Username and Password not accepted"

Question

I'm creating a Twitter bot and I'm implementing a method that sends me a email if there is an error. As I'm already using the google API to access Google Drive (have no problem here), I decided to use the service account to send the email (Google console says it could be used that way)

The method I've come up to send the email so far is:

var config = require('./config/mail');
var google = require('./config/google');
var nodemailer = require('nodemailer');

var send = function (args) {
  let transporter = nodemailer.createTransport({
    'service': 'gmail',
    'auth': {
        'type': 'OAuth2',
        'user': google.client_email,
        'serviceClient': google.client_id,
        'privateKey': google.private_key
    }
  });
  transporter.on('token', token => console.log(token));

  let message = {
    'from': `"${config.serverFromName}" <${config.serverFromMail}>`,
    'to': args.to,
    'subject': args.subject,
    'text': args.text,
    'html': `<p>${args.text}</p>`
  };

  transporter.sendMail(message, (err, info) => {
    if (err) {
      console.log('Mail couldn\'t be sent because: ' + err);
    } else {
      console.log('Mail sent');
    }
  });
};

The config/google file contains the data that Google generates for you when you create a service account. config.serverFromName and config.serverFromMail are the name and email of the sender (not the same as the service account id). args contains the recipent email and the content

When I test the send method, I got the following message in my console:

Mail couldn't be sent because: Error: Invalid login: 535-5.7.8 Username and Password not accepted. Learn more at
535 5.7.8  https://support.google.com/mail/?p=BadCredentials z123sm543690vkd.10 - gsmtp

I know the token is being created correctly because the listener I created is printing it:

{ user: '[email protected]',
  accessToken: 'ya29.ElmIBLxzfU_kkuZeyISeuRBeljmAe7HNTlwuG4K12ysUNo46s-eJ8NkMYHQqD_JrqTlH3yheNc2Aopu9B5vw-ivEqvPR4sTDpWBOg3xUU_4XiJEBLno8FHsg',
  expires: 1500151434603 }

Searching on the Internet I found that it may be a problem with the OAuth scope. However, all the info that talks about it refers to using Client IDs, not service accounts. I don't find that option in the Google developer console, either.

Any ideas of what I'm doing wrong?

Answer 1

Bottom Line: The specific way Google describes a service account is INCOMPATIBLE with nodemailer. BUT there is a way!

I have just spent countless hours myself up over this same issue! I have come to the conclusion, Google's Admin Console has removed half this capability indirectly. The console does not provide a way to authorize (a user accepting the consent screen) the desired scope the very first time with a service account.

First up, follow the Node.JS Quickstart instructions for Google Drive API to authorize a scope and receive a refresh token.

1. Go to console.developers.google.com, build a OAuth2.0 Client Id, and download the client_secret.json file.

2. Create a separate temporary module folder and use NPM to download google api modules

   npm install googleapis

   npm install google-auth-library

3. Create a quickstart.js file

4. Place your client_secret.json file next to quickstart.js

5. Line 7 in the quickstart.js is the array to define the scopes you intend to allow the application to access. Modify it as you see necessary. It is highly recommended to only provision access for what is intended. See Gmail API Scopes.

6. RUN node quickstart.js

7. Open the URL in a browser, authenticate, and copy the code from the browser back into the terminal window. This will download a nodejs-gmail-quickstart.json file which the location will be provided in stdout.
   This is the part you are unable to accomplish for a Service Account. This action authorizes the scopes provided in the SCOPES array to the downloaded access_token & refresh token.

NOTE: access_token's have a lifespan of 1 hour. refresh_token's are immortal.

Now you have an authorized refresh_token! Next is setting up your auth object with 3LO in Nodemailer. I would look more at the bottom examples because not all values are required. My auth looks like this:

const mailbot = nodemailer.createTransport({
      host: 'smtp.gmail.com',
      port: 587,              // TLS (google requires this port for TLS)
      secure: false,          // Not SSL
      requireTLS: true,       // Uses STARTTLS command (nodemailer-ism)
      auth: {
          // **HIGHLY RECOMMEND** ALL values be
          //  read in from a file not placed directly in code.  
          // Make sure that file is locked down to only the server daemon
          type : 'OAuth2',
          user : config.client_email,
          scope : "https://www.googleapis.com/auth/gmail.send",
          clientId : config.client_id,
          clientSecret: secret,
          refreshToken: activeToken.refresh_token

          // AT RUNTIME, it looks like this:
          //type : 'OAuth2',
          //user : '[email protected]',   // actual user being impersonated
          //scope : "", //Optional, but recommend to define for the action intended
          //clientId : '888888888998-9xx9x99xx9x99xx9xxxx9xx9xx9x88x8xxx.apps.googleusercontent.com',
          //clientSecret: 'XxxxxXXxX0xxxxxxxx0XXxX0',
          //refreshToken: '1/XXxXxsss-xxxXXXXXxXxx0XXXxxXXx0x00xxx'              
      }
 });

TIP: Gmail will rewrite the FROM field from any email sent with the authorized user account (user impersonated). If you want to customize this slightly, use the syntax { FROM: '"Display NAME" <user email>' } and it will not overwrite your display name choice since the email matches.

NOTE: nodemailer will make a token request out to https://accounts.google.com/o/oauth2/token with the refresh token to automatically obtain an access_token.

Unfortunately, nodemailer lacks the functionality to save a received token out to a file directly but instead just uses this.emit(). If the server stays active it will not be an issue but as mine is only bursting, it will always incur a delay as a new access_token will be requested every time.

[SECURITY] Hopefully this works for you! It is disappointing to loose the private key encryption a service account with 2LO would bring but at least this Client ID way is very hard to spoof. I was concerned about security but reading more I am okay with this implementation. See Google Identity Platform (Nodemailer uses the HTTP/REST details) and given

[1] Google's OAuth 2.0 endpoint is at https://accounts.google.com/o/oauth2/v2/auth. This endpoint is accessible only over HTTPS. Plain HTTP connections are refused.

[5] After the web server receives the authorization code, it can exchange the authorization code for an access token.

you are using TLS to connect initially for an authorization code, then matching it with your client ID data, and a refresh_token (you must go through the hassle we did above) then you can receive an access_token to actually interact with Google APIs.

As long as you increase your security posture with keeping the OAuth2.0 Client ID (highly random username), secret, and refresh token as separate, secure, and hidden as much as possible, you should be able to sleep soundly. GOOD LUCK!

Answer 2

After visiting the OAuth 2.0 Playground and experimenting with all possible variations of gmail-related sub-scopes, even selecting them altogether...

https://www.googleapis.com/auth/gmail.labels
https://www.googleapis.com/auth/gmail.send
https://www.googleapis.com/auth/gmail.readonly
https://www.googleapis.com/auth/gmail.compose
https://www.googleapis.com/auth/gmail.insert
https://www.googleapis.com/auth/gmail.modify
https://www.googleapis.com/auth/gmail.metadata
https://www.googleapis.com/auth/gmail.settings.basic
https://www.googleapis.com/auth/gmail.settings.sharing

...the error message described in the OP title still persist:

Error: Invalid login: 535-5.7.8 Username and Password not accepted

It seems that NodeMailer is not capable of connecting via the scopes mentioned above. In fact, it explicitly mentions in the "Troubleshooting" section of its OAuth2 SMTP transport docs

The correct OAuth2 scope for Gmail SMTP is https://mail.google.com/, make sure your client has this scope set when requesting permissions for an user

Although this gives access to more than just sending emails, it works!

The only alternative to reach a more fine grained scope solution seems to be to resort to google's own Gmail API, where you can pass scopes when generating the OAuth2 client (which should of course at least include the scopes granted at the time the OAuth consent screen was shown):

oAuth2Client.generateAuthUrl({
  access_type: 'offline',
  scope: SCOPES,
})