Stuck in WAITING_FOR_RUNNER while using an AMI for EC2 Resource

Question

I was able to successfully run Data pipeline for predefined templates. I wanted to use a customized AMI for my EC2 Resource as I require some libraries and packages to be installed.

I also have to run a python script as a part of the process.

So, I have created a Base Image of EC2 Resource with all packages required and the code that has to be run.

As part of my activity, I trigger a shell command activity, where I execute the python script as the command that has to be run.

The EC2 resource comes up successfully based on the Customized AMI that I have specified. I am able to login to that machine using the key pair that I specified but just that the Activity gets stuck in "WAITING FOR RUNNER" state.

I am not sure as to how to solve this problem. Please let me know if there are better ways to fix the same. Am I missing some basic step while trying to use a EC2 resource from an AMI.

Answer 1

Use Amazon Linux when creating your custom AMI and it will resolve this issue. Their OS comes preinstalled with tools that are used by data pipelines to communicate with the instance.