Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to run multiple beanstalk worker using php

Tags:

php

beanstalkd

As of now, I am running only one beanstalk worker thread for my project, which handles both cron based jobs and real time jobs. So, I want to separate it out into two workers. One worker is used to track the cron based jobs and one more to track the real time asynchronous jobs. So that, the worker efficiency will get improve. Anyone can help me on this,

  1. How to run and deamonize multiple beanstalk workers using php?
  2. A sample script to handle multiple beanstalk workers?

NOTE: Currently I am using pheanstalk php lib.

like image 283
karuh24 Avatar asked Apr 09 '11 12:04

karuh24


1 Answers

With pheanstalk, (or other libraries), if you want to accept jobs from a number of queues - just watch them.

$pheanstalk->watch('testtube')
           ->watch('tube2')
           ->watch('tube3');
$pheanstalk->reserve();  // get the next job from any of the tubes (+ 'default')

As for handling workers, I'm currently using Supervisord for some very similar scripts that I want to keep running. Its a python based daemon where the script you want to run is listed in a very simple configuration file. (adding more workers is literally changing a single number, and reloading the configuration).

One thing I have done is based on the blog post running-the-worker. Supervisord runs a shell script. That script runs the PHP which in turn returns a value (with exit($x)). If I return a value from the queue-runner, (for example), 99, I quit the shell script to shut down the worker. Another value may instantly restart the worker, Anything else, sleeps for a few seconds, before it restarts.

like image 68
Alister Bulman Avatar answered Nov 04 '22 16:11

Alister Bulman