Laravel schedules on distributed app

Question

My app runs on 3 servers behind a load balancer. Therefore it's stateless, all data is stored in redis and MySQL.

If my machines are running the artisan cron scheduler I'm assuming the same task will be run 3 times. Once on each machine, as there is no knowledge of what's been run shared between them aka a database table.

What are the solutions?

Answer 1

edit: In recent versions of Laravel, this is now built-in.

$schedule->command('foo:bar')->onOneServer();

We do this in Artisan commands that should only run on one server per cron execution:

public function handle()
    if(!Cache::add(get_class($this), true, 0.5)) {
        return false;
    }

Cache::add is synchronous and will return false if the key already exists, so even if all three servers execute at the exact same microsecond only one server will continue to the rest of the task.

Another common option is enabling the cron on just a single server in the cluster, or having a dedicated server for cron itself.

Answer 2

If your redis server is shared across the servers, you can create some logic like electing a leader appserver.

Your app logic should check the leader before processing. Sample snippet:

if(redis.get("LEADER").equals(currentappserver))
process();
else if(redis.get("LEADER")==null && selectLeader()) -> a lua script call.
process();

Since redis is single threaded write a logic like this snippet in a lua script:

selectLeader()
if redis.get("LEADER")==null
redis.set("LEADER",current_node)
return true;
else
return false;

It will be atomic and only one leader thread will be available always.

P.s: Codes I have provided are vauge pseudo code not the exact code.