AWS CredentialProviders fail to retrieve credentials in Fargate

Question

I'm running a SpringBoot app in AWS Fargate that uses SecretsManager. This is the what I supply to the AWS SDK as credential providers:

public class ProfiledCredentialsProvider extends AWSCredentialsProviderChain {

    public ProfiledCredentialsProvider(@Nullable final String profile) {
        super(new DefaultAWSCredentialsProviderChain(), new EC2ContainerCredentialsProviderWrapper(),
                new EnvironmentVariableCredentialsProvider(), new SystemPropertiesCredentialsProvider(),
                StringUtils.isBlank(profile) ? new ProfileCredentialsProvider()
                        : new ProfileCredentialsProvider(profile));
        this.setReuseLastProvider(true);
    }

}

and this allows me to run this application locally with an alternative AWS profile. When I run this app in Fargate though, I get the following stacktrace:

com.amazonaws.SdkClientException: Unable to load AWS credentials from any provider in the chain: [com.amazonaws.auth.DefaultAWSCredentialsProviderChain@3439f68d: Unable to load AWS credentials from any provider in the chain: [EnvironmentVariableCredentialsProvider: Unable to load AWS credentials from environment variables (AWS_ACCESS_KEY_ID (or AWS_ACCESS_KEY) and AWS_SECRET_KEY (or AWS_SECRET_ACCESS_KEY)), SystemPropertiesCredentialsProvider: Unable to load AWS credentials from Java system properties (aws.accessKeyId and aws.secretKey), WebIdentityTokenCredentialsProvider: To use assume role profiles the aws-java-sdk-sts module must be on the class path., com.amazonaws.auth.profile.ProfileCredentialsProvider@1cab0bfb: profile file cannot be null, com.amazonaws.auth.EC2ContainerCredentialsProviderWrapper@140e5a13: Failed to connect to service endpoint: ], com.amazonaws.auth.EC2ContainerCredentialsProviderWrapper@dbd940d: Failed to connect to service endpoint: , EnvironmentVariableCredentialsProvider: Unable to load AWS credentials from environment variables (AWS_ACCESS_KEY_ID (or AWS_ACCESS_KEY) and AWS_SECRET_KEY (or AWS_SECRET_ACCESS_KEY)), SystemPropertiesCredentialsProvider: Unable to load AWS credentials from Java system properties (aws.accessKeyId and aws.secretKey), com.amazonaws.auth.profile.ProfileCredentialsProvider@71d15f18: profile file cannot be null]
    at com.amazonaws.auth.AWSCredentialsProviderChain.getCredentials(AWSCredentialsProviderChain.java:136) ~[aws-java-sdk-core-1.11.793.jar!/:na]
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.getCredentialsFromContext(AmazonHttpClient.java:1257) ~[aws-java-sdk-core-1.11.793.jar!/:na]
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.runBeforeRequestHandlers(AmazonHttpClient.java:833) ~[aws-java-sdk-core-1.11.793.jar!/:na]
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:783) ~[aws-java-sdk-core-1.11.793.jar!/:na]
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:770) ~[aws-java-sdk-core-1.11.793.jar!/:na]
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:744) ~[aws-java-sdk-core-1.11.793.jar!/:na]
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:704) ~[aws-java-sdk-core-1.11.793.jar!/:na]
    at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:686) ~[aws-java-sdk-core-1.11.793.jar!/:na]
    at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:550) ~[aws-java-sdk-core-1.11.793.jar!/:na]
    at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:530) ~[aws-java-sdk-core-1.11.793.jar!/:na]
    at com.amazonaws.services.secretsmanager.AWSSecretsManagerClient.doInvoke(AWSSecretsManagerClient.java:2634) ~[aws-java-sdk-secretsmanager-1.11.793.jar!/:na]
    at com.amazonaws.services.secretsmanager.AWSSecretsManagerClient.invoke(AWSSecretsManagerClient.java:2601) ~[aws-java-sdk-secretsmanager-1.11.793.jar!/:na]
    at com.amazonaws.services.secretsmanager.AWSSecretsManagerClient.invoke(AWSSecretsManagerClient.java:2590) ~[aws-java-sdk-secretsmanager-1.11.793.jar!/:na]
    at com.amazonaws.services.secretsmanager.AWSSecretsManagerClient.executeGetSecretValue(AWSSecretsManagerClient.java:1213) ~[aws-java-sdk-secretsmanager-1.11.793.jar!/:na]
    at com.amazonaws.services.secretsmanager.AWSSecretsManagerClient.getSecretValue(AWSSecretsManagerClient.java:1184) ~[aws-java-sdk-secretsmanager-1.11.793.jar!/:na]

This is an extract of my task-definition.json:

{
  "family": "transfer-services-api",
  "executionRoleArn": "arn:aws:iam::************:role/ecs-task-execution-role"  
  "requiresCompatibilities": [
    "FARGATE"
  ]
}

with this in the 'trust relationship':

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "",
      "Effect": "Allow",
      "Principal": {
        "Service": "ecs-tasks.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

and attached policy AmazonECSTaskExecutionRolePolicy (no permission boundary set). Any help would be great, thanks.

Answer 1

You need to assign a task role. The execution role is what gives ECS access to resources like ECR and SecretsManager in order to execute your ECS task. The task role is what gives your task's code access to other AWS resources. See the documentation here.