Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

An enterprise scheduler for python (like quartz)

I am looking for an enterprise tasks scheduler for python, like quartz is for Java. Requirements:

  • Persistent: if the process restarts or the machine restarts, then all the jobs must stay there and must be fired after restarting.
  • Jobs must enter and exit the scheduler in a transaction (i.e. if some database operation fail, in a database unrelated to the scheduler, then the job must not have exited or entered the scheduler).
  • Scalability. Depends on the measure of success of the project, but I would prefer to know from the beggining that I am not starting from a dead end.
  • Configurability: when tasks expire, how many can be fired simultaneously, etc.

Any recommendations? Is there something python specific, or is it possible (or even good) to interface to Quartz from python.

like image 556
flybywire Avatar asked Nov 13 '09 04:11

flybywire


People also ask

Does Python have a scheduler?

Schedule is in-process scheduler for periodic jobs that use the builder pattern for configuration. Schedule lets you run Python functions (or any other callable) periodically at pre-determined intervals using a simple, human-friendly syntax.

What is Quartz Scheduler used for?

Quartz is a job scheduling library that can be integrated into a wide variety of Java applications. Quartz is generally used for enterprise class applications to support process workflow, system management (maintenance) actions and to provide timely services within the applications.

Does spring scheduler use Quartz?

Based on the docs, Spring provides three way of scheduling: @Scheduled. Via Quartz. Via JDK Timer.

How do I schedule an action in Python?

To schedule a Python script with Task scheduler, create an action and add the path to your Python executable file, add the path to the script in the “Start in” box and add the name of the Python file ase an argument. Then, create a trigger to schedule the execution of your script.


3 Answers

Is APScheduler what you're looking for?

like image 184
inspectorG4dget Avatar answered Sep 19 '22 18:09

inspectorG4dget


You can use Celery

Celery is an asynchronous task queue/job queue based on distributed message passing. It is focused on real-time operation, but supports scheduling as well.

Install Celery using pip install celery

Another option is using RQ.

RQ (Redis Queue) is a simple Python library for queueing jobs and processing them in the background with workers. It is backed by Redis and it is designed to have a low barrier to entry. It should be integrated in your web stack easily.

Install using pip install rq.

like image 37
user Avatar answered Sep 20 '22 18:09

user


We're using Sun Grid Engine, which is accessible through DRMAA, which happens to have Python bindings.

like image 36
Edward Dale Avatar answered Sep 18 '22 18:09

Edward Dale