How to setup celery as daemon

Question

I am faced with necessity to setup celery as daemon for my django project on Ubuntu 16.04 server while doing it I met several misunderstandings which I will describe in my question. I know that by the rules of Stack asked should should ask only one clear question but I will ask several in one question because first question come from second etc.

For tune celery as daemon I decide to use SystemD. In documentation Demonization celery provide a guide but it isn't so clear as I want maybe it's because I am beginner. My first question is: should I setup separately celery and celerybeat? Here is documentation provided example of configuration

[Unit] 
Description=Celery Service    
After=network.target 

[Service] 
Type=forking 
User=celery 
Group=celery 
EnvironmentFile=/etc/conf.d/celery 
WorkingDirectory=/opt/celery 
ExecStart=/bin/sh -c '${CELERY_BIN}    multi start ${CELERYD_NODES} \ -A ${CELERY_APP} --pidfile=${CELERYD_PID_FILE} \ --logfile=${CELERYD_LOG_FILE} --loglevel=${CELERYD_LOG_LEVEL} ${CELERYD_OPTS}' 
ExecStop=/bin/sh -c '${CELERY_BIN} multi stopwait ${CELERYD_NODES} \ --pidfile=${CELERYD_PID_FILE}'  
ExecReload=/bin/sh -c '${CELERY_BIN} multi restart ${CELERYD_NODES} \ -A ${CELERY_APP} --pidfile=${CELERYD_PID_FILE} \ --logfile=${CELERYD_LOG_FILE} --loglevel=${CELERYD_LOG_LEVEL} ${CELERYD_OPTS}' 

[Install] WantedBy=multi-user.target

All my misunderstandings relate to this piece of code and next question is:

What user and group I should specify in Service part of configuration? in documentation example above celery specified for user and group but when I checked users and groups on my Ubuntu I haven't celery user and group should I create this kind of user and group if answer is yes what rights I should provide for celery user and group? Or I should specify in user and group settings my Ubuntu user under which I am working on server?

Next question is related to all variables which used to specify different parts of settings for example in above configuration example in ExecStart settings have multiple variables

${CELERY_BIN}   
${CELERYD_NODES} 
${CELERY_APP} 
${CELERYD_PID_FILE} 
${CELERYD_LOG_FILE} 
${CELERYD_LOG_LEVEL} 
${CELERYD_OPTS}' 

Where I should specify this variables I think that it should be in django settings.py file but documentation above provide example which confused me maybe you can think why I can't to try configure this variables in settings.py rather than asking it here but I can't to test it because I don't have understanding about user and group question

Next question is about this setting

EnvironmentFile=/etc/conf.d/celery

What I should specify here because I have not this file on my system? Should I create it? It's seems that is default settings for celery but I use django and all related settings to celery I specified in my django project settings.py file

Can anyone guide me? Thank in advance

Answer 1

I too struggled to understand and implement the instructions given in celeryproject.org's daemonisation guide for systemd. Finally I just used general daemonisation instructions and it worked. I am using Flask, but I suppose this should work for you also :

This is my celery.service file. I have flask and celery running in a virtual environment (venv) :

[Unit]
Description=Celery Service
After=network.target

[Service]
User=your_username
WorkingDirectory=/home/your_username/your_projdir
Environment="PATH=/home/your_username/your_projdir/venv/bin"
ExecStart=/home/your_username/your_projdir/venv/bin/celery worker -A celery_worker.celery --loglevel=info

[Install]
WantedBy=default.target

You could add more commands here, and choose to include EnvironmentFile, but this simple setup worked well for me.