Starting multiple upstart instances automatically

Question

We use PHP gearman workers to run various tasks in parallel. Everything works just fine, and I have silly little shell script to spin them up when I want them. Being a programmer (and therefore lazy), I wanted to see if I could spin these up via an upstart script.

I figured out how to use the instance stanza, so I could start them with an instance number:

description "Async insert workers" author      "Mike Grunder"  env SCRIPT_PATH="/path/to/my/script"  instance $N  script     php $SCRIPT_PATH/worker.php end script 

And this works great, to start them like so:

sudo start async-worker N=1 sudo start async-worker N=2 

The way I want to use these workers is to spin up some number of them (maybe one per core, etc), and I would like to do this on startup. To be clear, I don't need the upstart script to detect the number of cores. I'm happy to just say "do 8 instances", but that's why I want multiple running. Is there a way for me to use the "start on" clause in an upstart script to do this automatically?

For example, start instance 1, 2, 3, 4? Then have them exit on shutdown properly?

I suppose I could hook this into an init.d script, but I was wondering if upstart can handle something like this, or if anyone has figured out this issue.

Cheers guys!

Answer 1

What you need is a bootstrap task that runs on startup and iterates over all your worker jobs, starting each one.

#/etc/init/async-workers-all.conf  start on runlevel [2345]  task  env NUM_WORKERS=8  script   for i in `seq 1 $NUM_WORKERS`   do     start async-worker N=$i   done end script 

The key is to make this a task, which tells upstart to let the task run to completion before emitting any events for it. See http://upstart.ubuntu.com/cookbook/#task and http://upstart.ubuntu.com/cookbook/#instance