spring data flow : IAM role assignment to pods using pod-annotations

Question

We are currently in the process of deploying a new spring data flow stream application in our aws EKS cluster. As part of this, the pods launched by the skipper should have the IAM roles defined in the annotation so that they can access the required AWS services. I have created the required iam role in AWS account and trying to pass the role using the pod-annotations property in the deploy stream for the application,

sample deployment property:

deployer.datastreamdemosource.kubernetes.pod-annotations=iam.amazonaws.com/role: arn:aws:iam::XXXXXXXX:role/spring-dataflow-test-role

The skipper is able to launch pods in the cluster but I can see the requested annotation is not assigned to the pod template,

The annotation value assigned to the pod is

Am i using the correct property to assign the required iam role? or how to assign IAM roles to the pods launched by the skipper as part of the stream?. We do not want to use a global IAM role for the streams since some pods will have additional permissions that they do not need. has anyone successfully deployed dataflow streaming applications in AWS EKS with iam roles for each pods ?

Answer 1

After reading documents for hours I found the required configurations in the below link

https://docs.spring.io/spring-cloud-dataflow/docs/current/reference/htmlsingle/#_annotations

So you need to use the below property definitions under the deployment Platform in your Streams which will allow the skipper to launch the pods with required annotations.

 deployer.data-stream-demo-source.kubernetes.podAnnotations=iam.amazonaws.com/role: arn:aws:iam::XXXXXXX:role/spring-dataflow-test-role
 deployer.data-stream-demo-sink.kubernetes.podAnnotations=iam.amazonaws.com/role: arn:aws:iam::XXXXXX:role/spring-dataflow-test-role

The default pod-annotation property in the deploy stream definition is not working as expected and not sure it's because of a typo in pod-annotations.

its the same case with other annotations as well (job-annotations,service-annotations).

For now, you can define the required annotation definition in the freetext tab or you can define in the application property level when registering apps to the dataflow server. By the way, I am using spring-cloud-dataflow-server Version: 2.2.1.RELEASE