How to run celery as a daemon in production?

Question

i created a celeryd file in /etc/defaults/ from the code here:

https://github.com/celery/celery/blob/3.0/extra/generic-init.d/celeryd

Now when I want to run celeryd as a daemon and do this: sudo /etc/init.d/celerdy it says command not found. Where am I going wrong?

Answer 1

I am not sure what you are doing here but these are the steps to run celery as a daemon.

1. The file that you have referred in the link https://github.com/celery/celery/blob/3.0/extra/generic-init.d/celeryd needs to be copied in your /etc/init.d folder with the name celeryd
2. Then you need to create a configuration file in the folder /etc/default with the name celeryd that is used by the above script. This configuration file basically defines certain variables and paths that are used by the above script. Here's an example configuration.
3. This link Generic init scripts explains the process and can be used for reference

Answer 2

I found this link extremely useful: How to write an Ubuntu Upstart job for Celery (django-celery) in a virtualenv

tweaking it a bit.. I have a celery worker running using this script:

(using ubuntu upstart)

named iamcelery.conf and placed it in /etc/init (note: not init.d)

# iamcelery -runs the celery worker as my virtual env user
#
#
# This task is run on startup to start the celery worker as my vritual env user

description "runs the celery worker"
author "michel van Leeuwen <[email protected]>"

start on runlevel [2345]
stop on runlevel [!2345]

# retry if ended unexpectedly
respawn
# limit the retries to max 15 times with timeouts of 5 seconds
respawn limit 15 5

# Time to wait between sending TERM and KILL signals
kill timeout 20

task
script
  exec su -s /bin/sh -c 'exec "$0" "$@"' <place here your unprovilegd username> -- srv/<here the path of your django project>/bin/django celeryd -BE -l info
end script

now you can start this scipt (it starts on server startup as well):

sudo start iamcelery

or stop:

sudo stop iamcelery

or check its status:

sudo status iamcelery

I am not quit sure this is the neatest way.... however... after a long trial and errors trying to get the initd scripts to work.... ( without succes) ... this finally works.

Edit 8 june 2013 My script given here seemed to runs as a root in the end. Now I changed this:

script
  su <place here your unprovilegd username>
  cd /srv/<here the path of your django project>/
  exec bin/django celeryd -BE -l info
end script

into:

script
  exec su -s /bin/sh -c 'exec "$0" "$@"' <place here your unprovilegd username> -- srv/<here the path of your django project>/bin/django celeryd -BE -l info
end script

and this works, with all the credits to the answer to this question: How to write an Ubuntu Upstart job for Celery (django-celery) in a virtualenv

Edit 5 sept 2013

There is one small thing left: I have to do ctrl-c after the start command in the console (and do a status check after this one): In case somebody knows this: leave in the command, and I can update this answer...