How to use Heroku Worker jobs with custom Clock with PHP programs?

Question

I am new to Heroku. I am trying to achieve something simple (at least I hope) but it seems there is no direct way to do so.

I have a couple of PHP files that I am trying to run as a scheduled job (every 5 mins). These runs some times can take more than 30 secs to complete. Hence as I understand I need to set them as worker jobs.

The example given on Heroku seems to be a complicated example from which I wasn't able to understand how an existing PHP file should be marked as worker.

Also, I would like to schedule this worker job to run every 5 mins. The docs refer to custom clock but there isn't an example for PHP.

Can someone please point me to a tutorial on how to use Worker processes and scheduling them Heroku for PHP programs? Something a beginner can follow.

Thank you!

Answer 1

The job of a clock process in this case is to call some other piece of code, lets call it a scheduler, at a regular, periodic interval. For the sake of argument lets say our scheduler is going to expect to be called every minute. That's the way the laravel scheduler and, hence, heroku clock I use (and wrote) are set up.

What are we trying to achieve here? For a clock to be a good clock it has to have some measure of reliability. We're not talking about microsecond accuracy or anything of the sort. My own design criteria is simply that the clock must call the scheduler every minute. That's all I need. In practice it does so within about the first five seconds of each minute according to the tests I've done (my clock code is at the foot of this answer - but don't look yet!).

Your clock process on heroku is going to have a procfile entry like clock: php artisan clock. This is in the case of laravel. That just runs the php command that contains the clock.

What does clock do? All it does is call scheduler, regularly, once a minute.

What does scheduler do? It provides a way to specify when you want jobs to run, (daily, weekly, 2pm on Tuesdays, etc.). When the scheduler is called, all it has to do is check in the list of all the schedulings you've asked it to do, and ask "Is there anything I need to do this particular minute?". If it finds anything, it runs it (your clock/scheduler dyno should dispatch to worker rather than run the job directly itself, to help it stay reliable) otherwise, it says "no tasks are scheduled to run at this time", and exits. Then the clock calls it again the next minute, and so on.

I realise the OP specified php and didn't mention laravel, apologies if this answer ends up being a bit laravel specific but it's what I know. Here's the scheduler command in the case of laravel. That provides you all the code you need to do the scheduling, all you need is to call it minutely. Enter the clock. The non-heroku way to set that up is add one cron entry that calls it every minute. On Heroku we just use a clock process that serves exactly the same purpose - just calls that command once a minute.

Thus, your task, should you choose to accept it, is to write a tiny little php program which you will run as a daemon (all heroku dynos specified in the Procfile run as daemons, i.e., continuously) (a procfile entry that wasn't a daemon would have little point to it), whose job it is is to sit there running, probably sleep for a bit, then wake up, and arrange itself such that it calls your scheduler, e.g. $this->call('schedule:run'), once a minute. That's all the clock needs to do, the rest is up to the scheduler program, whatever yours happens to be.

To write the actual clock, that's a fun little task, and you would definitely benefit from framing this problem yourself and thinking about it before you study my answer!

Here's what I'm using for my clock daemon (environment specific logging removed). The command that contains this code as its body is what's called in the Procfile as clock.

while(true) {
  if (now()->second <= 10)  // "runnable window"
  {
      $this->call('schedule:run');
      sleep(4);             // "min. duration"
  } 
  sleep(7);                 // "polling delay"
}

You should think about this and test it yourself, by just logging instead of calling schedule to start with, and see it in operation in your logs. This works so long as clock/scheduler process does not take more than say 30 seconds. You can achieve that by dispatching tasks from it to your worker, and not doing long running jobs in the clock (as the heroku docs advise), or it'll miss the next minute, because the clock will still be tied up running the scheduler from the previous minute.

Note dynos do reboot every day. So I let this run & checked it called reliably every minute even across cycling of the clock dyno. You can see that the code allows up to about max 10 second outage of the clock process to cover dyno restart. I don't know for sure what the max dyno outage on heroku is, it seemed to be a lot less than 10 seconds. These intervals worked well for me and I'm confident that in my use case (nothing absolutely critical), that this has reliability that's fine. It would be possible to set up more thorough tests than I have in fact done. I'd be interested to hear results if anyone does that. Set your clock running and make it log when it calls, & watch its behaviour when you push code & on the daily restart to give you some confidence of how it's behaving. I might write it up better than this if I get time! In the meantime, hope this helps. This does depend on the exact way heroku cycles its dynos, hence the importance of testing, that's how I see it at least. There may be a better algorithm for this, if someone has one, I'm all ears.

Finally, to answer the OP, you could set up a 'clock' that calls once every five minutes, but my suspicion is you might be better off setting up a clock that calls every one minute and calling a scheduler from it that simply asks 'is current time in seconds modulo five seconds equal to zero', then dispatches the job to the worker if it is. That means you can test your clock and we can develop and share results of a generic solution here.

Answer 2

For the clock process I solved in an easy way with the following three steps:

• Set credit card information in Heroku account;
• Installed Heroku Scheduler addon (you can use the command heroku addons:create scheduler:standard --app <yourAppName>);
• Set up the script to run as scheduled job:

More info here or here.