Menu
I have a Spring Boot app which communicates with S3 and SQS. It worked fine using AWS secret keys and secrets, but I found out I have a restriction in that I cannot use those credentials, but must instead authenticate using an IAM Instance Role.
I'm not having luck making this slight change work.
I've created an IAM Policy to allow my user to access the S3 bucket and SQS queue, here it is:
fooPolicy.json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:*"
],
"Resource": [
"arn:aws:s3:::foo-demo-bucket"
]
},
{
"Effect": "Allow",
"Action": [
"sqs:*"
],
"Resource": [
"arn:aws:sqs:::mysqsqueue"
]
}
]
}
Then I created an IAM Role using that policy and created a trust relationship for the role allowing foouser to assume that role
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::1234567890:user/foouser",
"Service": "ec2.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}
I'm running the app from a bash terminal where I have configured the aws cli so that from its perspective, I am logged in as foouser and foouser has been added to the role as a trusted entity.
However, when I run my app, as configured, I get the error:
... The security token included in the request is invalid. ...
java -Dconfig.file=./src/main/resources/application.yml -jar ./target/demo-0.0.1-SNAPSHOT.jar
. ____ _ __ _ _
/\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
\\/ ___)| |_)| | | | | || (_| | ) ) ) )
' |____| .__|_| |_|_| |_\__, | / / / /
=========|_|==============|___/=/_/_/_/
:: Spring Boot :: (v2.1.7.RELEASE)
2019-08-14 15:39:07.223 INFO 58892 --- [ main] com.example.demo.DemoApplication : Starting DemoApplication v0.0.1-SNAPSHOT on A6485192 with PID 58892 (/Users/foo/bar/src/demos3/target/demo-0.0.1-SNAPSHOT.jar started by foo in /Users/foo/bar/src/demos3)
2019-08-14 15:39:07.225 INFO 58892 --- [ main] com.example.demo.DemoApplication : No active profile set, falling back to default profiles: default
2019-08-14 15:39:10.785 INFO 58892 --- [ main] faultConfiguringBeanFactoryPostProcessor : No bean named 'errorChannel' has been explicitly defined. Therefore, a default PublishSubscribeChannel will be created.
2019-08-14 15:39:10.790 INFO 58892 --- [ main] faultConfiguringBeanFactoryPostProcessor : No bean named 'taskScheduler' has been explicitly defined. Therefore, a default ThreadPoolTaskScheduler will be created.
2019-08-14 15:39:10.793 INFO 58892 --- [ main] faultConfiguringBeanFactoryPostProcessor : No bean named 'integrationHeaderChannelRegistry' has been explicitly defined. Therefore, a default DefaultHeaderChannelRegistry will be created.
2019-08-14 15:39:10.810 INFO 58892 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'credentialsProvider' of type [com.amazonaws.auth.DefaultAWSCredentialsProviderChain] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2019-08-14 15:39:10.824 INFO 58892 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'org.springframework.integration.config.IntegrationManagementConfiguration' of type [org.springframework.integration.config.IntegrationManagementConfiguration$$EnhancerBySpringCGLIB$$364c7eab] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2019-08-14 15:39:10.838 INFO 58892 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'integrationDisposableAutoCreatedBeans' of type [org.springframework.integration.config.annotation.Disposables] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2019-08-14 15:39:11.206 WARN 58892 --- [ main] c.a.a.p.i.BasicProfileConfigLoader : Your profile name includes a 'profile ' prefix. This is considered part of the profile name in the Java SDK, so you will need to include this prefix in your profile name when you reference this profile from your Java code.
2019-08-14 15:39:12.729 WARN 58892 --- [ main] s.c.a.AnnotationConfigApplicationContext : Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'simpleMessageListenerContainer' defined in class path resource [org/springframework/cloud/aws/messaging/config/annotation/SqsConfiguration.class]: Invocation of init method failed; nested exception is com.amazonaws.services.sqs.model.AmazonSQSException: The security token included in the request is invalid. (Service: AmazonSQS; Status Code: 403; Error Code: InvalidClientTokenId; Request ID: 0b676d6d-5b41-5535-9d31-38a3d491aba6)
2019-08-14 15:39:12.735 INFO 58892 --- [ main] ConditionEvaluationReportLoggingListener :
Error starting ApplicationContext. To display the conditions report re-run your application with 'debug' enabled.
2019-08-14 15:39:12.740 ERROR 58892 --- [ main] o.s.boot.SpringApplication : Application run failed
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'simpleMessageListenerContainer' defined in class path resource [org/springframework/cloud/aws/messaging/config/annotation/SqsConfiguration.class]: Invocation of init method failed; nested exception is com.amazonaws.services.sqs.model.AmazonSQSException: The security token included in the request is invalid. (Service: AmazonSQS; Status Code: 403; Error Code: InvalidClientTokenId; Request ID: 0b676d6d-5b41-5535-9d31-38a3d491aba6)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1778) ~[spring-beans-5.1.9.RELEASE.jar!/:5.1.9.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:593) ~[spring-beans-5.1.9.RELEASE.jar!/:5.1.9.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:515) ~[spring-beans-5.1.9.RELEASE.jar!/:5.1.9.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:320) ~[spring-beans-5.1.9.RELEASE.jar!/:5.1.9.RELEASE]
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222) ~[spring-beans-5.1.9.RELEASE.jar!/:5.1.9.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:318) ~[spring-beans-5.1.9.RELEASE.jar!/:5.1.9.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199) ~[spring-beans-5.1.9.RELEASE.jar!/:5.1.9.RELEASE]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:845) ~[spring-beans-5.1.9.RELEASE.jar!/:5.1.9.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:877) ~[spring-context-5.1.9.RELEASE.jar!/:5.1.9.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:549) ~[spring-context-5.1.9.RELEASE.jar!/:5.1.9.RELEASE]
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:743) [spring-boot-2.1.7.RELEASE.jar!/:2.1.7.RELEASE]
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:390) [spring-boot-2.1.7.RELEASE.jar!/:2.1.7.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:312) [spring-boot-2.1.7.RELEASE.jar!/:2.1.7.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1214) [spring-boot-2.1.7.RELEASE.jar!/:2.1.7.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1203) [spring-boot-2.1.7.RELEASE.jar!/:2.1.7.RELEASE]
at com.example.demo.DemoApplication.main(DemoApplication.java:13) [classes!/:0.0.1-SNAPSHOT]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_212]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_212]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_212]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_212]
at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48) [demo-0.0.1-SNAPSHOT.jar:0.0.1-SNAPSHOT]
at org.springframework.boot.loader.Launcher.launch(Launcher.java:87) [demo-0.0.1-SNAPSHOT.jar:0.0.1-SNAPSHOT]
at org.springframework.boot.loader.Launcher.launch(Launcher.java:51) [demo-0.0.1-SNAPSHOT.jar:0.0.1-SNAPSHOT]
at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:52) [demo-0.0.1-SNAPSHOT.jar:0.0.1-SNAPSHOT]
Caused by: com.amazonaws.services.sqs.model.AmazonSQSException: The security token included in the request is invalid. (Service: AmazonSQS; Status Code: 403; Error Code: InvalidClientTokenId; Request ID: 0b676d6d-5b41-5535-9d31-38a3d491aba6)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1660) ~[aws-java-sdk-core-1.11.415.jar!/:na]
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1324) ~[aws-java-sdk-core-1.11.415.jar!/:na]
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1074) ~[aws-java-sdk-core-1.11.415.jar!/:na]
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:745) ~[aws-java-sdk-core-1.11.415.jar!/:na]
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:719) ~[aws-java-sdk-core-1.11.415.jar!/:na]
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:701) ~[aws-java-sdk-core-1.11.415.jar!/:na]
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:669) ~[aws-java-sdk-core-1.11.415.jar!/:na]
at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:651) ~[aws-java-sdk-core-1.11.415.jar!/:na]
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:515) ~[aws-java-sdk-core-1.11.415.jar!/:na]
at com.amazonaws.services.sqs.AmazonSQSClient.doInvoke(AmazonSQSClient.java:2147) ~[aws-java-sdk-sqs-1.11.415.jar!/:na]
at com.amazonaws.services.sqs.AmazonSQSClient.invoke(AmazonSQSClient.java:2116) ~[aws-java-sdk-sqs-1.11.415.jar!/:na]
at com.amazonaws.services.sqs.AmazonSQSClient.invoke(AmazonSQSClient.java:2105) ~[aws-java-sdk-sqs-1.11.415.jar!/:na]
at com.amazonaws.services.sqs.AmazonSQSClient.executeGetQueueUrl(AmazonSQSClient.java:1138) ~[aws-java-sdk-sqs-1.11.415.jar!/:na]
at com.amazonaws.services.sqs.AmazonSQSClient.getQueueUrl(AmazonSQSClient.java:1110) ~[aws-java-sdk-sqs-1.11.415.jar!/:na]
at org.springframework.cloud.aws.messaging.support.destination.DynamicQueueUrlDestinationResolver.resolveDestination(DynamicQueueUrlDestinationResolver.java:94) ~[spring-cloud-aws-messaging-2.1.2.RELEASE.jar!/:2.1.2.RELEASE]
at org.springframework.cloud.aws.messaging.support.destination.DynamicQueueUrlDestinationResolver.resolveDestination(DynamicQueueUrlDestinationResolver.java:38) ~[spring-cloud-aws-messaging-2.1.2.RELEASE.jar!/:2.1.2.RELEASE]
at org.springframework.messaging.core.CachingDestinationResolverProxy.resolveDestination(CachingDestinationResolverProxy.java:92) ~[spring-messaging-5.1.9.RELEASE.jar!/:5.1.9.RELEASE]
at org.springframework.cloud.aws.messaging.listener.AbstractMessageListenerContainer.queueAttributes(AbstractMessageListenerContainer.java:320) ~[spring-cloud-aws-messaging-2.1.2.RELEASE.jar!/:2.1.2.RELEASE]
at org.springframework.cloud.aws.messaging.listener.AbstractMessageListenerContainer.initialize(AbstractMessageListenerContainer.java:292) ~[spring-cloud-aws-messaging-2.1.2.RELEASE.jar!/:2.1.2.RELEASE]
at org.springframework.cloud.aws.messaging.listener.SimpleMessageListenerContainer.initialize(SimpleMessageListenerContainer.java:111) ~[spring-cloud-aws-messaging-2.1.2.RELEASE.jar!/:2.1.2.RELEASE]
at org.springframework.cloud.aws.messaging.listener.AbstractMessageListenerContainer.afterPropertiesSet(AbstractMessageListenerContainer.java:267) ~[spring-cloud-aws-messaging-2.1.2.RELEASE.jar!/:2.1.2.RELEASE]
at org.springframework.cloud.aws.messaging.listener.SimpleMessageListenerContainer.afterPropertiesSet(SimpleMessageListenerContainer.java:45) ~[spring-cloud-aws-messaging-2.1.2.RELEASE.jar!/:2.1.2.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1837) ~[spring-beans-5.1.9.RELEASE.jar!/:5.1.9.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1774) ~[spring-beans-5.1.9.RELEASE.jar!/:5.1.9.RELEASE]
... 23 common frames omitted
Thank you to anyone who can help me solve this
Here is the source for the app that demonstrates my problem.
DemoApplication.java
package com.example.demo;
import lombok.extern.slf4j.Slf4j;
import org.springframework.boot.CommandLineRunner;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
@Slf4j
@SpringBootApplication
public class DemoApplication implements CommandLineRunner {
public static void main(String[] args) {
SpringApplication.run(DemoApplication.class, args);
}
@Override
public void run(String... args) throws Exception {
while(true) {
Thread.sleep(1000);
}
}
}
application.yml
cloud:
aws:
stack:
auto: false
credentials:
accessKey:
secretKey:
instanceProfile: true
useDefaultAwsCredentialsChain: true
region:
static: us-east-1
aws:
enabled: true
region: us-east-1
user: foouser
access-key:
secret-key:
sqs:
queue: mysqsqueue
s3:
bucket: foo-demo-bucket
AWS.java
package com.example.demo;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.model.PutObjectResult;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.cloud.aws.messaging.listener.annotation.SqsListener;
import org.springframework.stereotype.Component;
import java.io.File;
@Slf4j
@Component
public class AWS {
@Autowired
private AmazonS3 amazonS3;
@Value("${aws.s3.bucket}")
private String bucket;
PutObjectResult upload(String filePath, String uploadKey) {
File file = new File(filePath);
return amazonS3.putObject(bucket, uploadKey, file);
}
@SqsListener("mysqsqueue")
public void queueListener(String message) {
System.out.println("Got an SQS message: " + message);
}
}
AWSConfiguration.java
package com.example.demo;
import com.amazonaws.auth.AWSCredentials;
import com.amazonaws.auth.AWSCredentialsProvider;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3ClientBuilder;
import com.amazonaws.services.sqs.AmazonSQSAsync;
import com.amazonaws.services.sqs.AmazonSQSAsyncClientBuilder;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.cloud.aws.messaging.config.annotation.EnableSqs;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Primary;
@Configuration
@EnableSqs
public class AWSConfiguration {
@Value("${aws.region}")
private String awsRegion;
@Value("${aws.access-key}")
private String awsAccessKey;
@Value("${aws.secret-key}")
private String awsSecretKey;
@Bean
@Primary
public AmazonSQSAsync amazonSQSAsyncClient() {
AmazonSQSAsync amazonSQSAsyncClient = AmazonSQSAsyncClientBuilder.standard()
.withCredentials(amazonAWSCredentials())
.withRegion(awsRegion)
.build();
return amazonSQSAsyncClient;
}
@Bean
public AmazonS3 amazonS3Client() {
AmazonS3 s3Client = AmazonS3ClientBuilder.standard()
.withCredentials(amazonAWSCredentials())
.withRegion(awsRegion).build();
return s3Client;
}
@Bean
@Primary
public AWSCredentialsProvider amazonAWSCredentials() {
return new AWSCredentialsProvider() {
public void refresh() {}
public AWSCredentials getCredentials() {
return new AWSCredentials() {
public String getAWSSecretKey() {
return awsSecretKey;
}
public String getAWSAccessKeyId() {
return awsAccessKey;
}
};
}
};
}
}
987
IAM roles. An IAM role is very similar to a user, in that it is an identity with permission policies that determine what the identity can and cannot do in AWS. However, a role does not have any credentials (password or access keys) associated with it.
Assuming a role involves using a set of temporary security credentials that you can use to access AWS resources that you might not normally have access to. These temporary credentials consist of an access key ID, a secret access key, and a security token.
Users as service accounts An IAM user is a resource in IAM that has associated credentials and permissions. An IAM user can represent a person or an application that uses its credentials to make AWS requests. This is typically referred to as a service account.
Try to use STSAssumeRoleSessionCredentialsProvider to login and get credentials using role
@Value("${cloud.aws.assumeRoleARN:}")
private String assumeRoleARN;
@Autowired
private AWSCredentialsProvider awsCredentialsProvider;
@Bean
@Primary
public AWSCredentialsProvider awsCredentialsProvider() {
log.info("Assuming role {}",assumeRoleARN);
if (StringUtils.isNotEmpty(assumeRoleARN)) {
AWSSecurityTokenService stsClient = AWSSecurityTokenServiceClientBuilder.standard()
.withClientConfiguration(clientConfiguration())
.withCredentials(awsCredentialsProvider)
.build();
return new STSAssumeRoleSessionCredentialsProvider
.Builder(assumeRoleARN, "test")
.withStsClient(stsClient)
.build();
}
return awsCredentialsProvider;
}
Your IAM policy is wrong - the SQS resource should include wildcards for AWS region and account id (arn:aws:sqs:*:*:mysqsqueue instead of arn:aws:sqs:::mysqsqueue).
The policy below should work.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:*"
],
"Resource": [
"arn:aws:s3:::foo-demo-bucket"
]
},
{
"Effect": "Allow",
"Action": [
"sqs:*"
],
"Resource": [
"arn:aws:sqs:*:*:mysqsqueue"
]
}
]
}
It's always good to confirm whether your IAM policies are specified correctly using IAM simulator as it catches most silly mistakes.
44
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
