Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Scheduling a regular event: Cron/Cron alternatives (including Celery)

Something I've had interest in is regularly running a certain set of actions at regular time intervals. Obviously, this is a task for cron, right?

Unfortunately, the Internet seems to be in a bit of disagreement there.

Let me elaborate a little about my setup. First, my development environment is in Windows, while my production environment is hosted on Webfaction (Linux). There is no real cron on Windows, right? Also, I use Django! And what's suggested for Django?

Celery of course! Unfortunately, setting up Celery has been more or less a literal nightmare for me - please see Error message 'No handlers could be found for logger “multiprocessing”' using Celery. And this is only ONE of the problems I've had with Celery. Others include a socket error which it I'm the only one ever to have gotten the problem.

Don't get me wrong, Celery seems REALLY cool. Unfortunately, there seems to be a lack of support, and some odd limitations built into its preferred backend, RabbitMQ. Unfortunately, no matter how cool a program is, if it doesn't work, well, it doesn't work!

That's where I hope all of you can come in. I'd like to know about cron or a cron-equivalent, which can be set up similarly (preferably identically) in both a Windows and a Linux environment.

(I've been struggling with Celery for about two weeks now and unfortunately I think it's time to toss in the towel and give up on it, at least for now.)

like image 700
RHH Avatar asked Jun 08 '11 12:06

RHH


People also ask

How do you schedule celery tasks?

A task is just a Python function. You can think of scheduling a task as a time-delayed call to the function. For example, you might ask Celery to call your function task1 with arguments (1, 3, 3) after five minutes. Or you could have your function batchjob called every night at midnight.

How do I set a cron schedule?

Cron jobs are scheduled at recurring intervals, specified using a format based on unix-cron. You can define a schedule so that your job runs multiple times a day, or runs on specific days and months. (Although we no longer recommend its use, the legacy App Engine cron syntax is still supported for existing jobs.)

How do I schedule a cron job in Python?

First, create your Python script. Then, open the system terminal your working with. To access crontab, input 'crontab -e' (one will be created if it doesn't already exist). Then enter 'i' to initiate the edit mode, and proceed to input your schedule command.


3 Answers

I had the same problem, and held off trying to solve it with celery (too complicated) or cron (external to application) and ended up finding Advanced Python Scheduler. Only just started using it but it seems reasonably mature and stable, has decent documentation and will take a number of scheduling formats (e.g. cron style).

From the documentation, running a function at a specific interval.

from apscheduler.scheduler import Scheduler
sched = Scheduler()
sched.start()
def hello_world():
    print "hello world"
sched.add_interval_job(hello_world,seconds=10)

This is non-blocking, and I run something pretty identical by simply importing the module from my urls.py. Hope this helps.

like image 57
mrmagooey Avatar answered Oct 19 '22 22:10

mrmagooey


https://github.com/andybak/django-cron

Triggered by a single cron task but all the scheduling and configuration is done in Python.

like image 29
Andy Baker Avatar answered Oct 19 '22 23:10

Andy Baker


Django Chronograph is a great alternative. You only need to setup one cron then do everything in django admin. You can schedule tasks/commands from django management.

like image 43
Noel Pure Avatar answered Oct 19 '22 22:10

Noel Pure