InvalidClientTokenId: The security token included in the request is invalid. status code: 403

Question

I am using, terraform & kubectl to deploy insfra-structure and application.

Since I changed aws configure :

terraform init

terraform apply

I always got :

terraform apply

Error: error validating provider credentials: error calling sts:GetCallerIdentity: InvalidClientTokenId: The security token included in the request is invalid.
    status code: 403, request id: 5ba38c31-d39a-11e9-a642-21e0b5cf5c0e

  on providers.tf line 1, in provider "aws":
   1: provider "aws" {

Can you advise ? Appreciate !

Answer 1

From here.

This is a general error that can be cause by a few reasons.

Some examples:

1) Invalid credentials passed as environment variables or in ~/.aws/credentials.

Solution: Remove old profiles / credentials and clean all your environment vars:

for var in AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY AWS_SESSION_TOKEN AWS_SECURITY_TOKEN ; do eval unset $var ; done

2) When your aws_secret_access_key contains characters like the plus-sign + or multiple forward-slash /. See more in here.
Solution: Delete credentials and generate new ones.

3) When you try to execute Terraform inside a region which must be explicitly enabled (and wasn't).
(In my case it was me-south-1 (Bahrain) - See more in here).
Solution: Enable region or move to an enabled one.

4) In cases where you work with 3rd party tools like Vault and don't supply valid AWS credentials to communicate with - See more in here.

All will lead to a failure of aws sts:GetCallerIdentity API.

Answer 2

I got the same invalid token error after adding an S3 Terraform backend.

It was because I was missing a profile attribute on the new backend.

This was my setup when I got the invalid token error:

# ~/.aws/credentials

[default]
aws_access_key_id=OJA6...
aws_secret_access_key=r2a7...

[my_profile_name]
aws_access_key_id=RX9T...
aws_secret_access_key=oaQy...

// main.tf

terraform {
  backend "s3" {
    bucket         = "terraform-state"
    encrypt        = true
    key            = "terraform.tfstate"
    region         = "us-east-1"
    dynamodb_table = "terraform-state-locks"
  }
}

And this was the fix that worked (showing a diff, I added the line with "+" at the beginning):

  // main.tf

  terraform {
    backend "s3" {
      bucket         = "terraform-state"
      // ...
+     profile        = "my_profile_name"
    }
  }

None of the guides or videos I read or watched included the profile attribute. But it's explained in the Terraform documentation, here:

https://www.terraform.io/language/settings/backends/s3

Answer 3

In my case, it turned out that I had the environment variables AWS_ACCESS_KEY_ID, AWS_DEFAULT_REGION and AWS_SECRET_ACCESS_KEY set. This circumvented my ~/.aws/credentials file. Simply unsetting these environment variables worked for me!