Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Introducing randomness of time using cron and task queue

I'm looking for some engineering creativity to solve a problem on Google App Engine.

I have a small number of jobs that run periodically, but I'd like the jobs to be executed at random times. So instead of running a job every Tuesday at 2:00pm, I'd like it to run every Tuesday "between 2:00pm and 5:00pm".

Currently, I'm using the following algorithm...

  1. Cron job runs every Tuesday at 2:00pm
  2. The cron handler finds a list of specific jobs to run and creates a task queue event for each discrete task.
  3. The respective task queue handler decides if it should actually run by picking a random number between one and N. If the random number is X, the job gets executed. Otherwise, it creates a new task queue event to try again. Each task has a maximum number of queue attempts to guarantee that the job actually completes at some point.

I've realized that another solution would be to create a task queue that has a very slow rate, and when the cron job fills the queue, it randomly re-orders the list of tasks before doing so.

Any ideas from App Engine users?

like image 902
Greg Avatar asked Apr 11 '11 00:04

Greg


1 Answers

Have a cron job at 2 pm that queues a task with a random countdown between 0 and 3 hours?

like image 155
Bemmu Avatar answered Nov 15 '22 07:11

Bemmu