How to configure this Spring-Boot app to use IAM Role instead of keys and secrets?

Question

I have a Spring Boot app which communicates with S3 and SQS. It worked fine using AWS secret keys and secrets, but I found out I have a restriction in that I cannot use those credentials, but must instead authenticate using an IAM Instance Role.

I'm not having luck making this slight change work.

I've created an IAM Policy to allow my user to access the S3 bucket and SQS queue, here it is:

fooPolicy.json

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:*"
      ],
      "Resource": [
        "arn:aws:s3:::foo-demo-bucket"
      ]
    },
    {
      "Effect": "Allow",
      "Action": [
        "sqs:*"
      ],
      "Resource": [
        "arn:aws:sqs:::mysqsqueue"
      ]
    }
  ]
}

Then I created an IAM Role using that policy and created a trust relationship for the role allowing foouser to assume that role

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::1234567890:user/foouser",
        "Service": "ec2.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

I'm running the app from a bash terminal where I have configured the aws cli so that from its perspective, I am logged in as foouser and foouser has been added to the role as a trusted entity.

However, when I run my app, as configured, I get the error:

... The security token included in the request is invalid. ...

 java -Dconfig.file=./src/main/resources/application.yml -jar ./target/demo-0.0.1-SNAPSHOT.jar

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::        (v2.1.7.RELEASE)

2019-08-14 15:39:07.223  INFO 58892 --- [           main] com.example.demo.DemoApplication         : Starting DemoApplication v0.0.1-SNAPSHOT on A6485192 with PID 58892 (/Users/foo/bar/src/demos3/target/demo-0.0.1-SNAPSHOT.jar started by foo in /Users/foo/bar/src/demos3)
2019-08-14 15:39:07.225  INFO 58892 --- [           main] com.example.demo.DemoApplication         : No active profile set, falling back to default profiles: default
2019-08-14 15:39:10.785  INFO 58892 --- [           main] faultConfiguringBeanFactoryPostProcessor : No bean named 'errorChannel' has been explicitly defined. Therefore, a default PublishSubscribeChannel will be created.
2019-08-14 15:39:10.790  INFO 58892 --- [           main] faultConfiguringBeanFactoryPostProcessor : No bean named 'taskScheduler' has been explicitly defined. Therefore, a default ThreadPoolTaskScheduler will be created.
2019-08-14 15:39:10.793  INFO 58892 --- [           main] faultConfiguringBeanFactoryPostProcessor : No bean named 'integrationHeaderChannelRegistry' has been explicitly defined. Therefore, a default DefaultHeaderChannelRegistry will be created.
2019-08-14 15:39:10.810  INFO 58892 --- [           main] trationDelegate$BeanPostProcessorChecker : Bean 'credentialsProvider' of type [com.amazonaws.auth.DefaultAWSCredentialsProviderChain] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2019-08-14 15:39:10.824  INFO 58892 --- [           main] trationDelegate$BeanPostProcessorChecker : Bean 'org.springframework.integration.config.IntegrationManagementConfiguration' of type [org.springframework.integration.config.IntegrationManagementConfiguration$$EnhancerBySpringCGLIB$$364c7eab] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2019-08-14 15:39:10.838  INFO 58892 --- [           main] trationDelegate$BeanPostProcessorChecker : Bean 'integrationDisposableAutoCreatedBeans' of type [org.springframework.integration.config.annotation.Disposables] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2019-08-14 15:39:11.206  WARN 58892 --- [           main] c.a.a.p.i.BasicProfileConfigLoader       : Your profile name includes a 'profile ' prefix. This is considered part of the profile name in the Java SDK, so you will need to include this prefix in your profile name when you reference this profile from your Java code.
2019-08-14 15:39:12.729  WARN 58892 --- [           main] s.c.a.AnnotationConfigApplicationContext : Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'simpleMessageListenerContainer' defined in class path resource [org/springframework/cloud/aws/messaging/config/annotation/SqsConfiguration.class]: Invocation of init method failed; nested exception is com.amazonaws.services.sqs.model.AmazonSQSException: The security token included in the request is invalid. (Service: AmazonSQS; Status Code: 403; Error Code: InvalidClientTokenId; Request ID: 0b676d6d-5b41-5535-9d31-38a3d491aba6)
2019-08-14 15:39:12.735  INFO 58892 --- [           main] ConditionEvaluationReportLoggingListener :

Error starting ApplicationContext. To display the conditions report re-run your application with 'debug' enabled.
2019-08-14 15:39:12.740 ERROR 58892 --- [           main] o.s.boot.SpringApplication               : Application run failed

org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'simpleMessageListenerContainer' defined in class path resource [org/springframework/cloud/aws/messaging/config/annotation/SqsConfiguration.class]: Invocation of init method failed; nested exception is com.amazonaws.services.sqs.model.AmazonSQSException: The security token included in the request is invalid. (Service: AmazonSQS; Status Code: 403; Error Code: InvalidClientTokenId; Request ID: 0b676d6d-5b41-5535-9d31-38a3d491aba6)
  at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1778) ~[spring-beans-5.1.9.RELEASE.jar!/:5.1.9.RELEASE]
  at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:593) ~[spring-beans-5.1.9.RELEASE.jar!/:5.1.9.RELEASE]
  at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:515) ~[spring-beans-5.1.9.RELEASE.jar!/:5.1.9.RELEASE]
  at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:320) ~[spring-beans-5.1.9.RELEASE.jar!/:5.1.9.RELEASE]
  at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222) ~[spring-beans-5.1.9.RELEASE.jar!/:5.1.9.RELEASE]
  at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:318) ~[spring-beans-5.1.9.RELEASE.jar!/:5.1.9.RELEASE]
  at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199) ~[spring-beans-5.1.9.RELEASE.jar!/:5.1.9.RELEASE]
  at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:845) ~[spring-beans-5.1.9.RELEASE.jar!/:5.1.9.RELEASE]
  at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:877) ~[spring-context-5.1.9.RELEASE.jar!/:5.1.9.RELEASE]
  at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:549) ~[spring-context-5.1.9.RELEASE.jar!/:5.1.9.RELEASE]
  at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:743) [spring-boot-2.1.7.RELEASE.jar!/:2.1.7.RELEASE]
  at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:390) [spring-boot-2.1.7.RELEASE.jar!/:2.1.7.RELEASE]
  at org.springframework.boot.SpringApplication.run(SpringApplication.java:312) [spring-boot-2.1.7.RELEASE.jar!/:2.1.7.RELEASE]
  at org.springframework.boot.SpringApplication.run(SpringApplication.java:1214) [spring-boot-2.1.7.RELEASE.jar!/:2.1.7.RELEASE]
  at org.springframework.boot.SpringApplication.run(SpringApplication.java:1203) [spring-boot-2.1.7.RELEASE.jar!/:2.1.7.RELEASE]
  at com.example.demo.DemoApplication.main(DemoApplication.java:13) [classes!/:0.0.1-SNAPSHOT]
  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_212]
  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_212]
  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_212]
  at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_212]
  at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48) [demo-0.0.1-SNAPSHOT.jar:0.0.1-SNAPSHOT]
  at org.springframework.boot.loader.Launcher.launch(Launcher.java:87) [demo-0.0.1-SNAPSHOT.jar:0.0.1-SNAPSHOT]
  at org.springframework.boot.loader.Launcher.launch(Launcher.java:51) [demo-0.0.1-SNAPSHOT.jar:0.0.1-SNAPSHOT]
  at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:52) [demo-0.0.1-SNAPSHOT.jar:0.0.1-SNAPSHOT]
Caused by: com.amazonaws.services.sqs.model.AmazonSQSException: The security token included in the request is invalid. (Service: AmazonSQS; Status Code: 403; Error Code: InvalidClientTokenId; Request ID: 0b676d6d-5b41-5535-9d31-38a3d491aba6)
  at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1660) ~[aws-java-sdk-core-1.11.415.jar!/:na]
  at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1324) ~[aws-java-sdk-core-1.11.415.jar!/:na]
  at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1074) ~[aws-java-sdk-core-1.11.415.jar!/:na]
  at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:745) ~[aws-java-sdk-core-1.11.415.jar!/:na]
  at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:719) ~[aws-java-sdk-core-1.11.415.jar!/:na]
  at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:701) ~[aws-java-sdk-core-1.11.415.jar!/:na]
  at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:669) ~[aws-java-sdk-core-1.11.415.jar!/:na]
  at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:651) ~[aws-java-sdk-core-1.11.415.jar!/:na]
  at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:515) ~[aws-java-sdk-core-1.11.415.jar!/:na]
  at com.amazonaws.services.sqs.AmazonSQSClient.doInvoke(AmazonSQSClient.java:2147) ~[aws-java-sdk-sqs-1.11.415.jar!/:na]
  at com.amazonaws.services.sqs.AmazonSQSClient.invoke(AmazonSQSClient.java:2116) ~[aws-java-sdk-sqs-1.11.415.jar!/:na]
  at com.amazonaws.services.sqs.AmazonSQSClient.invoke(AmazonSQSClient.java:2105) ~[aws-java-sdk-sqs-1.11.415.jar!/:na]
  at com.amazonaws.services.sqs.AmazonSQSClient.executeGetQueueUrl(AmazonSQSClient.java:1138) ~[aws-java-sdk-sqs-1.11.415.jar!/:na]
  at com.amazonaws.services.sqs.AmazonSQSClient.getQueueUrl(AmazonSQSClient.java:1110) ~[aws-java-sdk-sqs-1.11.415.jar!/:na]
  at org.springframework.cloud.aws.messaging.support.destination.DynamicQueueUrlDestinationResolver.resolveDestination(DynamicQueueUrlDestinationResolver.java:94) ~[spring-cloud-aws-messaging-2.1.2.RELEASE.jar!/:2.1.2.RELEASE]
  at org.springframework.cloud.aws.messaging.support.destination.DynamicQueueUrlDestinationResolver.resolveDestination(DynamicQueueUrlDestinationResolver.java:38) ~[spring-cloud-aws-messaging-2.1.2.RELEASE.jar!/:2.1.2.RELEASE]
  at org.springframework.messaging.core.CachingDestinationResolverProxy.resolveDestination(CachingDestinationResolverProxy.java:92) ~[spring-messaging-5.1.9.RELEASE.jar!/:5.1.9.RELEASE]
  at org.springframework.cloud.aws.messaging.listener.AbstractMessageListenerContainer.queueAttributes(AbstractMessageListenerContainer.java:320) ~[spring-cloud-aws-messaging-2.1.2.RELEASE.jar!/:2.1.2.RELEASE]
  at org.springframework.cloud.aws.messaging.listener.AbstractMessageListenerContainer.initialize(AbstractMessageListenerContainer.java:292) ~[spring-cloud-aws-messaging-2.1.2.RELEASE.jar!/:2.1.2.RELEASE]
  at org.springframework.cloud.aws.messaging.listener.SimpleMessageListenerContainer.initialize(SimpleMessageListenerContainer.java:111) ~[spring-cloud-aws-messaging-2.1.2.RELEASE.jar!/:2.1.2.RELEASE]
  at org.springframework.cloud.aws.messaging.listener.AbstractMessageListenerContainer.afterPropertiesSet(AbstractMessageListenerContainer.java:267) ~[spring-cloud-aws-messaging-2.1.2.RELEASE.jar!/:2.1.2.RELEASE]
  at org.springframework.cloud.aws.messaging.listener.SimpleMessageListenerContainer.afterPropertiesSet(SimpleMessageListenerContainer.java:45) ~[spring-cloud-aws-messaging-2.1.2.RELEASE.jar!/:2.1.2.RELEASE]
  at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1837) ~[spring-beans-5.1.9.RELEASE.jar!/:5.1.9.RELEASE]
  at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1774) ~[spring-beans-5.1.9.RELEASE.jar!/:5.1.9.RELEASE]
  ... 23 common frames omitted

Thank you to anyone who can help me solve this

Here is the source for the app that demonstrates my problem.

DemoApplication.java

package com.example.demo;

import lombok.extern.slf4j.Slf4j;
import org.springframework.boot.CommandLineRunner;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

@Slf4j
@SpringBootApplication
public class DemoApplication implements CommandLineRunner {

  public static void main(String[] args) {
    SpringApplication.run(DemoApplication.class, args);
  }

  @Override
  public void run(String... args) throws Exception {
    while(true) {
      Thread.sleep(1000);
    }
  }
}

application.yml

cloud:
  aws:
    stack:
      auto: false
    credentials:
      accessKey:
      secretKey:
      instanceProfile: true
      useDefaultAwsCredentialsChain: true
    region:
      static: us-east-1

aws:
  enabled: true
  region: us-east-1
  user: foouser
  access-key:
  secret-key:

  sqs:
    queue: mysqsqueue

  s3:
    bucket: foo-demo-bucket

AWS.java

package com.example.demo;

import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.model.PutObjectResult;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.cloud.aws.messaging.listener.annotation.SqsListener;
import org.springframework.stereotype.Component;
import java.io.File;

@Slf4j
@Component
public class AWS {

  @Autowired
  private AmazonS3 amazonS3;

  @Value("${aws.s3.bucket}")
  private String bucket;

  PutObjectResult upload(String filePath, String uploadKey) {
    File file = new File(filePath);
    return amazonS3.putObject(bucket, uploadKey, file);
  }

  @SqsListener("mysqsqueue")
  public void queueListener(String message) {
    System.out.println("Got an SQS message: " + message);
  }
}

AWSConfiguration.java

package com.example.demo;

import com.amazonaws.auth.AWSCredentials;
import com.amazonaws.auth.AWSCredentialsProvider;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3ClientBuilder;
import com.amazonaws.services.sqs.AmazonSQSAsync;
import com.amazonaws.services.sqs.AmazonSQSAsyncClientBuilder;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.cloud.aws.messaging.config.annotation.EnableSqs;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Primary;

@Configuration
@EnableSqs
public class AWSConfiguration {

  @Value("${aws.region}")
  private String awsRegion;

  @Value("${aws.access-key}")
  private String awsAccessKey;

  @Value("${aws.secret-key}")
  private String awsSecretKey;

  @Bean
  @Primary
  public AmazonSQSAsync amazonSQSAsyncClient() {
    AmazonSQSAsync amazonSQSAsyncClient = AmazonSQSAsyncClientBuilder.standard()
        .withCredentials(amazonAWSCredentials())
        .withRegion(awsRegion)
        .build();
    return amazonSQSAsyncClient;
  }

  @Bean
  public AmazonS3 amazonS3Client() {
    AmazonS3 s3Client = AmazonS3ClientBuilder.standard()
        .withCredentials(amazonAWSCredentials())
        .withRegion(awsRegion).build();
    return s3Client;
  }

  @Bean
  @Primary
  public AWSCredentialsProvider amazonAWSCredentials() {
    return new AWSCredentialsProvider() {
      public void refresh() {}
      public AWSCredentials getCredentials() {
        return new AWSCredentials() {
          public String getAWSSecretKey() {
            return awsSecretKey;
          }
          public String getAWSAccessKeyId() {
            return awsAccessKey;
          }
        };
      }
    };
  }
}

Answer 1

Try to use STSAssumeRoleSessionCredentialsProvider to login and get credentials using role

@Value("${cloud.aws.assumeRoleARN:}")
private String assumeRoleARN;

@Autowired
private AWSCredentialsProvider awsCredentialsProvider;

@Bean
@Primary
public AWSCredentialsProvider awsCredentialsProvider() {
    log.info("Assuming role {}",assumeRoleARN);
    if (StringUtils.isNotEmpty(assumeRoleARN)) {
        AWSSecurityTokenService stsClient = AWSSecurityTokenServiceClientBuilder.standard()
                .withClientConfiguration(clientConfiguration())
                .withCredentials(awsCredentialsProvider)
                .build();

        return new STSAssumeRoleSessionCredentialsProvider
                .Builder(assumeRoleARN, "test")
                .withStsClient(stsClient)
                .build();
    }
   return awsCredentialsProvider;
}

Answer 2

Your IAM policy is wrong - the SQS resource should include wildcards for AWS region and account id (arn:aws:sqs:*:*:mysqsqueue instead of arn:aws:sqs:::mysqsqueue).

The policy below should work.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:*"
      ],
      "Resource": [
        "arn:aws:s3:::foo-demo-bucket"
      ]
    },
    {
      "Effect": "Allow",
      "Action": [
        "sqs:*"
      ],
      "Resource": [
        "arn:aws:sqs:*:*:mysqsqueue"
      ]
    }
  ]
}

It's always good to confirm whether your IAM policies are specified correctly using IAM simulator as it catches most silly mistakes.