Are wordpress cron jobs executed sequentially or in parallel?

Question

I am trying to limit the number of emails sent from my website to cope with the hosting service email limitations. I am using cron jobs and an indicator of piling the emails in the database to check if the number of emails sent is approaching the limit of max emails sent.

The way I do that, is by directly executing the scheduled process then make it 'sleep' for a certain period of time (according to its position in the queue) and then send the email and log in the database. To explain further the reason why I am using scheduled tasks and 'sleep', consider the scenario below:

1. A user tries to register to my website and expects an email to be sent to him/her shortly. Thus, if the emails/minute quota is exceeded, I need to send a different message: "Our server is busy, please permit 'x' minutes to perform the required task".
2. The requests to send email are all done through AJAX. Using 'sleep' within the process itself is not an option because the user will have to wait the x minutes until the 'busy message' is echo'd.
3. I tried with ob_flush, flush...etc. combinations to echo the message then the server works out everything in the background, but that never worked. The AJAX call was always waiting for the script to end to echo the result.

4. I need multi-threading in the single-threaded PHP language! As a workaround I used cron jobs, where each piled email is scheduled to be executed at time() (i.e. directly fire the scheduled job) which is hooked to the same function that sends emails. Using a flag, the function knows that the request is a piled email and makes it 'sleep' until the time required for the email quota reset.

5. The problem: If 5 people registered at almost the same time (while we still have an email pile), then we have 5 cron jobs that should all be executed at the same time and then sleep for a while (the sleep time can differ if the number of emails in the pile are already greater than the email quota), then emails are sent. However, when I check the logs in the database, I find that the scheduled jobs are executed sequentially and not in parallel. Sometimes it happens a cron job is fired before the other ends, but they don't fire at the same time.

I know that wordpress cron jobs are not really cron jobs and are fired once somebody visits the website (and I make sure I refresh the pages after the registration requests are sent to fire all of the scheduled tasks), but they seem to be the only option for me since my hosting server doesn't allow access to the server neither allows scheduling cron jobs.

Here is part of the code that executes the above:

//Test example to pile up emails, where quota is set to 2 emails every 30 seconds
$Emails_Threshold = 2;
$Threshold_Duration = 0.5*60;
//Get email indicator info
$Email_Info = $wpdb->get_row( 
"SELECT * 
FROM PileEmails 
WHERE priority = -1 
AND Status='New';"
,ARRAY_A);

if ($sleep ==0 && $Queue_Info_id==0){ //Not a scheduled event
   //Check if there are queued emails
   $Queue_exist = $wpdb->get_row (
    $wpdb->prepare("
                    SELECT Status 
                    FROM PileEmails
                    WHERE Status='Queued';"
                    ,$mail_priority)
                ,ARRAY_A);
  if (!empty($Queue_exist) || ($Email_Info['last_email_time']   > (time()-$Threshold_Duration))){
    if ($Email_Info['count_emails']>=$Emails_Threshold){
           //Code to Pile up
    }
  }else{ 
   //Reset email counter
  }
}else{ 
$wpdb->insert( "PileEmails",$Sleep_Info,$format);
sleep(10); //10 seconds here just as an example
}
//Code to send emails

Here is what I get logged into the database when I try to send 10 emails after exceeding the quota.

Notice that the time stamp has 10 seconds difference between each log and the following one although they should all be fired at the same time and each sleeps for 10 seconds then all send emails in parallel.

So my question is: Why does wordpress cron fire the scheduled jobs sequentially and not in parallel? and how to overcome this problem?

Any help is much appreciated.

Answer 1

As mentioned, installing a cron plugin will help to manage your crons.

To answer your question, Wordpress uses a "cron lock" defined('DOING_CRON') and sets a transient $lock = get_transient('doing_cron') when the spawn_cron method is invoked.

So, if you have a look at wp-includes/cron.php you'll see Wordpress crons do not run concurrently by default and will not run more than once every 60 seconds.

// don't run if another process is currently running it or more than once every 60 sec.
    if ( $lock + WP_CRON_LOCK_TIMEOUT > $gmt_time )
        return;