Running airflow worker gives error: Address already in use

Question

I am running Airflow with CeleryExecutor. I am able to run the commands airflow webserver and airflow scheduler but trying to run airflow worker gives the error: socket.error: [Errno 98] Address already in use.

The traceback:

Answer 1

In the docker container running Airflow server a process was already running on the port 8793 which the worker_log_server_port settings in airflow.cfg refers by default. I changed the port to 8795 and the command airflow worker worked.

Or you can check the process listening to 8793 as: lsof i:8793 and if you don't need that process you kill it by: kill $(lsof -t -i:8793). I was running ubuntu container in docker I had to install lsof first:

apt-get update
apt-get install lsof

Answer 2

See if there's server_logs process running, if so, kill it and try again.

/usr/bin/python2 /usr/bin/airflow serve_logs