Setting up a scheduled / cron job with Django on Elastic Beanstalk with a Worker Tier

Question

I'm currently in the process of migrating a Django website from my own hosted server running Ubuntu to AWS Elastic Beanstalk.

I've found the process somewhat straight-forward so far - until trying to set up a few scheduled jobs for my app. From what I can gather, I want to run a cron job on a worker tier environment using a cron.yaml file. I've read through the docs: http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/using-features-managing-env-tiers.html#worker-periodictasks

And read the blog post: https://medium.com/@joelennon/running-cron-jobs-on-amazon-web-services-aws-elastic-beanstalk-a41d91d1c571#.mx7dq9ufo

And various StackOverflow posts, but I feel like I'm still missing some fundamental concepts about what actually makes up my worker tier environment. On my own server I could simply set up a cron job to fit this need - so this concept is rather new to me. I also have a few Django apps running on Heroku that use web and worker dynos, async processing, Redis and Celery and scheduled jobs, but I can't work out how to translate this into the Elastic Beanstalk world.

Basically, the concepts I want to understand are:

1. What actually makes up my worker tier environment as far as code goes? Obviously more than just the cron.yaml file. Is this an exact clone of my web app, deployed to this environment as well? Or can this somehow reference the code from my web environment and run that way?
2. Or is the worker app its own complete whole new app altogether? Do I need to create a separate full-blown Django / Flask app to do this?
3. How does my worker app physically talk to my web app? How is the POST messages in the cron.yaml actually meant to execute jobs on the web app? If it's a standalone app, how are the worker and web environments actually linked?

I essentially want to schedule some Django management commands. I've exposed methods as POST endpoints as well but can't figure out how to get the worker environment to talk to / execute jobs on the web app.

Excuse my naivety, I would really appreciate any sort of advice and direction on how this concept all comes together.

Answer 1

So I ended up talking to a friend who's more familiar with the AWS services. He explained the concepts, and I got the scheduled jobs running by setting up the worker environment as follows:

• Built a separate standalone application to the web environment. I built a separate "worker" Django app, but this could be Flask or really any other framework or language
• Created an app called "cron" that had views to handle POST messages to different endpoints, which are essentially the scheduled jobs I wanted to execute. These endpoints are what the jobs in my cron.yaml file direct to
• As my jobs needed to make database changes to the web app, I set up the worker app to use the same database as the web app. This was as simple as adding RDS environment variables to my worker environment configuration. Eg. Set RDS_DB_NAME, RDS_HOSTNAME, RDS_USERNAME to point to the web environment database

Et voila, the scheduled jobs executed on schedule and make the database changes as required.