boto3 sessions and aws_session_token management

Question

I am developing python software which deals with AWS SQS queues. It uses boto3, mostly boto3.session.Session.

I have seen here that we can pass an aws_session_token to the Session constructor.

When running my code outside of Amazon, I need to periodically refresh this aws_session_token since it is only valid for an hour. So I need to reinstantiate a boto3.Session on my own.

I am just wondering how things work inside AWS. Do I need to manually refresh my sessions by getting a new aws_session_token through the environment? Or is my session valid "for ever"/is it handled internally so I don't have to refresh my AWS sessions?

The documentation seems unclear to me.

Answer 1

Short answer:

AWS generated tokens do not last forever, and same goes for any boto3 session created with generated tokens. But you can set a lengthy TTL on your tokens (up to 36 hours) as long as your tokens weren't generated with the account root user. This gives you a lot of time to do what you need to do with your Python script.

AWS has several ways of handling temporary and permanent access to your account. Generally, you'll want to rely on temporary credentials, as they are safer to use and align more with best practices. Boto3 uses a prioritized list of where it scans for credentials described here

Long, rambling answer:

I write a lot of automation code for dozens of AWS accounts, so I've dealt with this stuff a lot.

This assumes you're developing in Linux. Windows is very similar, but has some differences.

There are (at least) three methods to handle remote access to your AWS account:

1. Maintain a profile in your ~/.aws/credentials file which contains your AWS IAM user access keys, and run your Python script using that profile.

   • All your Python script has to do is create a boto3.session.Session object with no parameters. When you don't provide tokens or a profile name for the session instanstiation, boto3 automatically looks for credentials by scanning through the credentials priority list described in the link above.

   • I don't recommend this at all, but it works and give you an idea of how AWS profiles are used. This is permanent access using your IAM user's API keys, which never expire. While you can use these keys for any action that your IAM user has been granted permission, you shouldn't use them for anything other than assuming specialized roles to do all other work.

2. Assume a role using the AWS CLI from the command line, load the tokens into environment variables, and then run your Python script.

   • Advantages:
     • Easily automated.
     • Can set the token TTL easily.
     • The tokens can be loaded into environment variables and become instantly available to your Python scripts.
   • Disadvantages:
     • Only practical if your Python script is interacting with one AWS account.
     • If your Python script runs longer than the token TTL (unlikely, but not impossible), then your script will hit an AccessDenied error and stop.

3. Run the Python script and have it handle role assumption and token juggling.

   • Advantages:
     • Allows your to juggle access to multiple account in one place.
     • Can set the token TTL easily.
     • If tokens expire, you can catch the AccessDened exception, refresh the tokens, and keep going.
   • Disadvantages:
     • Involves maintaining the Python code which gets the access tokens and creates boto sessions with them. Granted, it's not that much code, but its still code, which means maintenance and clutter.

I generally prefer method 2 and strongly discourage method 1. Method 3 is situational.

Method 1:
From the command line, set your AWS_PROFILE variable to your profile name and run the script. Everything done in the script with use your AWS profile (IAM user access keys).

AWS_PROFILE=<YOUR_CREDENTIALS_PROFILE_NAME> python <PATH_TO_SCRIPT>

Method 2:
From the command line, use your AWS profile to assume a role in the account, and then store the generated tokens in environment variables. Now when you execute the script, it will use those tokens automatically:

credentials=`AWS_PROFILE=<YOUR_AWS_PROFILE_NAME> aws sts assume-role --role-arn <YOUR_AWS_ROLE_NAME> --role-session-name <SOME_SESSION_NAME> --query 'Credentials.{AKI:AccessKeyId,SAK:SecretAccessKey,ST:SessionToken}' --output text`

export AWS_ACCESS_KEY_ID=`echo ${credentials} | awk '{print $1}'`
export AWS_SECRET_ACCESS_KEY=`echo ${credentials} | awk '{print $2}'`
export AWS_SECURITY_TOKEN=`echo ${credentials} | awk '{print $3}'`
export AWS_DEFAULT_REGION=<AWS_REGION>

python <path_to_your_python_script>

Note: since your tokens are loaded into environment variables, AWS_PROFILE should NOT be set when you run your script. All AWS SDKs automatically look for credential tokens in those environment variables. You can read more about them here.

Method 3:
In your Python code, generate the access tokens and then create a session with those tokens.

import boto3

role_info = {
    'RoleArn': 'arn:aws:iam::<AWS_ACCOUNT_NUMBER>:role/<AWS_ROLE_NAME>',
    'RoleSessionName': '<SOME_SESSION_NAME>'
}

client = boto3.client('sts')
credentials = client.assume_role(**role_info)

session = boto3.session.Session(
    aws_access_key_id=credentials['Credentials']['AccessKeyId'],
    aws_secret_access_key=credentials['Credentials']['SecretAccessKey'],
    aws_session_token=credentials['Credentials']['SessionToken']
)

Run your script the same as Method 1, except this time your AWS_PROFILE is used to assume the role and any subsequent work is performed through the role since the session is created with the assumed role.

AWS_PROFILE=<YOUR_CREDENTIALS_PROFILE_NAME> python <PATH_TO_SCRIPT>

Hope this helps!