Amazon EC2 user data script on instance startup - python script won't run?

Question

I am following the tutorial on Running Commands on Your Linux Instance at Launch - Amazon Elastic Compute Cloud.

I have an EC2 instance and I've confirmed that running

python run.py

does exactly what I want (write a file to s3) when I run it through the command line. So in the web UI I added

#!/bin/bash

python run.py

into the user data field. But when I start it back up nothing happens. I must be overlooking something, but not sure what it is.

One thing that did surprise me was that all the python packages and scripts are still there when I start the instance back up, I thought everything would be wiped, and I would have to copy the script over from s3 after each launch. But I guess that's just the storage? Like I said I'm new to this, so any help will be appreciated!

Answer 1

There's a very silly mistake we normally make with the user data scripts. We forget that they run from the root user. This means that the current directory while running the script would be the HOME directory of user "root" and not the home directory of Ubuntu user or ec2-user based on your OS.

This means if your command ran successfully, the file could be lying at a relevant HOME location. However, the user data script expects it to be in a different location. Trying giving an absolute path of the script and see if that works.

Answer 2

There was a notice in the link you mentioned

By default, user data scripts and cloud-init directives run only during the boot cycle when you first launch an instance. You can update your configuration to ensure that your user data scripts and cloud-init directives run every time you restart your instance.

and linked to this page for further details.

Alternatively you can simply add the job to crontab as follows (need to SSH to the EC2 instance):

@reboot python /PATH/TO/run.py