Find a better way to renew AWS credentials

Question

I am using sts:assumeRole to connect to a s3 bucket of a different account. Now, the job that I run takes a few days and along the way the credentials expire and I needed a way to renew them.

I have written the following code to handle expiry of the temporary credentials

This code is inside my downloadFile():

return new Promise((resolve, reject) => {
    function responseCallback(error, data) {
        if (error) {
            const errorMessage = `Fail to download file from s3://${config().s3.bucket}/${path}: ${error};`;
            reject(error);
        } else {
            Logger.info(`Successfully download file from s3://${config().s3.bucket}/${path}`);
            resolve(data.Body);
        }
    }
    const fn = this.s3Client.getObject({
        Bucket: config().s3.bucket,
        Key: path
    }, (error, data) => this.handleTokenExpiry(error, data, fn, responseCallback));
});

And this is the handleTokenExpiry()

handleTokenExpiry(error, data, fn, callback) {
    if (!error || error.code !== "ExpiredToken") return callback(error, data);

    Logger.info("Token expired, creating new token");
    this.s3Client = null; // null so that init() doesn't return existing s3Client
    return this.init().then(fn);
}

Here init() is the method which sets this.s3Client using sts:assumeRole and then new AWS.S3()

This works fine but I am not sure if this a clean way to do it. The strange thing is when I test it in local it takes almost two minutes for responseCallback() to be called when token is expired. Though responseCallback() gets executed immediately while the token is active.

Answer 1

The AWS SDK can handle refreshing the credentials for you. For example:

const credentials = new ChainableTemporaryCredentials({
  params: {
    RoleArn: "optional-role-arn",
    RoleSessionName: `required-parameter-${Date.now()}`
  }
})
const s3 = new AWS.S3({credentials})

Now AWS SDK will refresh the tokens behind the scenes without any action from caller of s3. For more information, please see AWSK SDK Documentation. Refresh is limited to validity time of the credentials used.

Answer 2

For tasks running less than 12h, here is the solution.

When using AssumeRole, you can specify DurationSeconds argument to specify the duration of the temporary credentials returned by STS. This is 15 min minimum, up to 12h.

The role you are assuming needs to be modified to authorize the maximum duration too. See https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use.html#id_roles_use_view-role-max-session

In your case, with a job running for several days. I would suggest to refactor the app to run in smaller batches, each running for a few hours each.

Another alternative would be to be proactive about token expiration. If your code know the token duration and the time at which it acquired the token, I would suggest to call a method before calling a method that uses the token (such as S3's getObject). That method you check if the token are soon to expire and proactively refresh them. Pseudo code would be like

function refreshToken() {
   return new Promise( (resolve, reject) => {
      // XX depends how long is your S3 getObject call
      if (token_acquisition_time + token_duration <= now() + xx minutes)  {
         // refresh token
         sts.assumeRole(...).promise().then(resolve());
      } else {
         resolve();
      }
   });
}

...

refreshToken.then(s3.getObject(...));