OSError: [Errno 13] Permission denied when initializing Celery in Docker

Question

I've been getting the following error when running docker compose. The problem doesn't happen at all in my Mac OS dev environment (this error in occuring when trying to deploy in Ubuntu & debian), but the error seems to suggest that Celery does not have access to write the celerybeat file. I've been trying for days trying to get this to work (trying to giving Celery the necessary permissions), but had no luck.

Error

celery_1    | [2017-06-17 13:08:26,509: INFO/Beat] beat: Starting...
celery_1    | [2017-06-17 13:08:26,556: ERROR/Beat] Removing corrupted schedule file 'celerybeat-schedule': DBAccessError(13, 'Permission denied')
celery_1    | Traceback (most recent call last):
celery_1    |   File "/usr/local/lib/python2.7/site-packages/celery/beat.py", line 376, in setup_schedule
celery_1    |     self._store = self._open_schedule()
celery_1    |   File "/usr/local/lib/python2.7/site-packages/celery/beat.py", line 366, in _open_schedule
celery_1    |     return self.persistence.open(self.schedule_filename, writeback=True)
celery_1    |   File "/usr/local/lib/python2.7/shelve.py", line 243, in open
celery_1    |     return DbfilenameShelf(filename, flag, protocol, writeback)
celery_1    |   File "/usr/local/lib/python2.7/shelve.py", line 227, in __init__
celery_1    |     Shelf.__init__(self, anydbm.open(filename, flag), protocol, writeback)
celery_1    |   File "/usr/local/lib/python2.7/anydbm.py", line 85, in open
celery_1    |     return mod.open(file, flag, mode)
celery_1    |   File "/usr/local/lib/python2.7/dbhash.py", line 18, in open
celery_1    |     return bsddb.hashopen(file, flag, mode)
celery_1    |   File "/usr/local/lib/python2.7/bsddb/__init__.py", line 364, in hashopen
celery_1    |     d.open(file, db.DB_HASH, flags, mode)
celery_1    | DBAccessError: (13, 'Permission denied')
celery_1    | [2017-06-17 13:08:26,558: ERROR/Beat] Process Beat
celery_1    | Traceback (most recent call last):
celery_1    |   File "/usr/local/lib/python2.7/site-packages/billiard/process.py", line 292, in _bootstrap
celery_1    |     self.run()
celery_1    |   File "/usr/local/lib/python2.7/site-packages/celery/beat.py", line 553, in run
celery_1    |     self.service.start(embedded_process=True)
celery_1    |   File "/usr/local/lib/python2.7/site-packages/celery/beat.py", line 470, in start
celery_1    |     humanize_seconds(self.scheduler.max_interval))
celery_1    |   File "/usr/local/lib/python2.7/site-packages/kombu/utils/__init__.py", line 325, in __get__
celery_1    |     value = obj.__dict__[self.__name__] = self.__get(obj)
celery_1    |   File "/usr/local/lib/python2.7/site-packages/celery/beat.py", line 512, in scheduler
celery_1    |     return self.get_scheduler()
celery_1    |   File "/usr/local/lib/python2.7/site-packages/celery/beat.py", line 507, in get_scheduler
celery_1    |     lazy=lazy)
celery_1    |   File "/usr/local/lib/python2.7/site-packages/celery/utils/imports.py", line 53, in instantiate
celery_1    |     return symbol_by_name(name)(*args, **kwargs)
celery_1    |   File "/usr/local/lib/python2.7/site-packages/celery/beat.py", line 358, in __init__
celery_1    |     Scheduler.__init__(self, *args, **kwargs)
celery_1    |   File "/usr/local/lib/python2.7/site-packages/celery/beat.py", line 185, in __init__
celery_1    |     self.setup_schedule()
celery_1    |   File "/usr/local/lib/python2.7/site-packages/celery/beat.py", line 384, in setup_schedule
celery_1    |     self._store = self._destroy_open_corrupted_schedule(exc)
celery_1    |   File "/usr/local/lib/python2.7/site-packages/celery/beat.py", line 371, in _destroy_open_corrupted_schedule
celery_1    |     self._remove_db()
celery_1    |   File "/usr/local/lib/python2.7/site-packages/celery/beat.py", line 363, in _remove_db
celery_1    |     os.remove(self.schedule_filename + suffix)
celery_1    | OSError: [Errno 13] Permission denied: 'celerybeat-schedule'

Excerpt from Dockerfile

FROM python:2.7-slim
MAINTAINER Maintainer <[email protected]>    

RUN apt-get -y install sudo

ENV INSTALL_PATH /minebase
RUN mkdir -p $INSTALL_PATH

WORKDIR $INSTALL_PATH

COPY requirements.txt requirements.txt
RUN pip install -r requirements.txt

COPY . .
RUN pip install --editable .

CMD gunicorn -c "python:config.gunicorn" "minebase.app:create_app()"

RUN groupadd -r celery && useradd -r -g celery celery | chpasswd && adduser celery sudo

RUN sudo chown -R celery ./
USER celery

CMD ["celery", "-A", "worker:app", "worker"]

Docker-compose.yml

version: '2'

services:

  postgres:
    image: 'postgres:9.5'
    env_file:
      - '.env'
    volumes:
      - 'postgres:/var/lib/postgresql/data'
    ports:
      - '5432:5432'

  redis:
    image: 'redis:3.0-alpine'
    command: redis-server --requirepass password
    volumes:
      - 'redis:/var/lib/redis/data'
    ports:
      - '6379:6379'

  minebase:
    build: .
    command: >
      gunicorn -c "python:config.gunicorn" --reload "minebase.app:create_app()" --timeout 7200 --workers=5
    env_file:
      - '.env'
    volumes:
      - '.:/minebase'
    ports:
      - '8000:8000'

  nginx:
    restart: always
    build: ./nginx/
    ports:
      - "80:80"
    volumes:
      - '.:/minebase'
    volumes_from:
      - minebase
    links:
      - minebase:minebase
    expose:
      - 80

  celery:
    build: .
    command: celery worker -B -l info -A minebase.blueprints.contact.tasks 
    env_file:
      - '.env'
    volumes:
      - '.:/minebase'

volumes:
  postgres:
  redis:

Versions

Celery version: v3.1.23

Docker version: 17.05.0-ce

Docker compose version: 1.13.0

Answer 1

You do not need all that magic with sudo and chown in Docker container.

Use existing unprivileged user nobody:

FROM some-base-image
WORKDIR /app
RUN chown nobody:nogroup "celerybeat-schedule"
USER nobody
CMD ["celery", "-A", "your_project.celery_app", "-E", "-B"]

Another approach is specifying a custom location for "celerybeat-schedule" file. For example, in /tmp directory:

FROM some-base-image
WORKDIR /app
USER nobody
CMD ["celery", "-A", "your_project.celery_app",, "-E", "-B", \
     "-s", "/tmp/celerybeat-schedule"]