EMR activity stuck in Waiting_For_Runner state

Question

I am creating a data pipeline to export dynamoDB table to S3 bucket.I used the standard template to use for this in data pipeline console. I ha verified that the runsOn field is set to the name of EMR cluster to be started. However, The EMR activity status is still as Waiting_For_Runner...Any ideas why is this so??

Thanks!!!

Answer 1

Waiting_For_Runner means datatpipeline is trying to connect to EMR.

Few reasons you can check:

1. IAM permissions between EMR and Data pipeline(Roles). Here's a link!
2. Check Task runner is running or not on master instance
   $ps -ef | grep workerGroup (Master Instance)
3. Check --workgroup name in EMR(you can see in Task runner process) and compare it with the name of workgroup in datapipeline.