Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Airflow: Can't connect to ('0.0.0.0', 8080)

Tags:

python

airflow

I am on Ubuntu 16.04,I have installed Airflow with pip. Next step

airflow initdb
[2017-07-29 12:20:23,483] {__init__.py:57} INFO - Using executor SequentialExecutor
DB: sqlite:////home/milenko/airflow/airflow.db
[2017-07-29 12:20:23,813] {db.py:287} INFO - Creating tables
INFO  [alembic.runtime.migration] Context impl SQLiteImpl.

This works fine.But when I try to start webserver

airflow webserver -p 8080
[2017-07-29 12:20:45,913] [4585] {models.py:167} INFO - Filling up the DagBag from /home/milenko/airflow/dags
Running the Gunicorn Server with:
Workers: 4 sync
Host: 0.0.0.0:8080
Timeout: 120
Logfiles: - -
=================================================================            
[2017-07-29 12:20:46,950] {__init__.py:57} INFO - Using executor SequentialExecutor
[2017-07-29 12:20:47 +0200] [4590] [INFO] Starting gunicorn 19.3.0
[2017-07-29 12:20:47 +0200] [4590] [ERROR] Connection in use: ('0.0.0.0', 8080)
[2017-07-29 12:20:47 +0200] [4590] [ERROR] Retrying in 1 second.
[2017-07-29 12:20:48 +0200] [4590] [ERROR] Connection in use: ('0.0.0.0', 8080)
[2017-07-29 12:20:48 +0200] [4590] [ERROR] Retrying in 1 second.
[2017-07-29 12:20:49 +0200] [4590] [ERROR] Connection in use: ('0.0.0.0', 8080)
[2017-07-29 12:20:49 +0200] [4590] [ERROR] Retrying in 1 second.
[2017-07-29 12:20:50 +0200] [4590] [ERROR] Connection in use: ('0.0.0.0', 8080)
[2017-07-29 12:20:50 +0200] [4590] [ERROR] Retrying in 1 second.
[2017-07-29 12:20:51 +0200] [4590] [ERROR] Connection in use: ('0.0.0.0', 8080)
[2017-07-29 12:20:51 +0200] [4590] [ERROR] Retrying in 1 second.
[2017-07-29 12:20:52 +0200] [4590] [ERROR] Can't connect to ('0.0.0.0', 8080)

How to solve this?Is this Gunicorn Server issue or not? With netstat I got this output

sudo netstat -nlp | grep :80
tcp6       0      0 :::8080                 :::*                    LISTEN      1711/java

As I understand the port is in use.But what is 1711/java?If I turn it off will it affect something else? This is the relevant part of my airflow configuration file

[webserver]
# The base url of your website as airflow cannot guess what domain or
# cname you are using. This is used in automated emails that
# airflow sends to point links to the right web server
base_url = http://localhost:8080

# The ip specified when starting the web server
web_server_host = 0.0.0.0

# The port on which to run the web server
web_server_port = 8080

# Paths to the SSL certificate and key for the web server. When both are
# provided SSL will be enabled. This does not change the web server port.
web_server_ssl_cert =
web_server_ssl_key =

# Number of seconds the gunicorn webserver waits before timing out on a worker
web_server_worker_timeout = 120

# Number of workers to refresh at a time. When set to 0, worker refresh is
# disabled. When nonzero, airflow periodically refreshes webserver workers by
# bringing up new ones and killing old ones.
worker_refresh_batch_size = 1

# Number of seconds to wait before refreshing a batch of workers.
worker_refresh_interval = 30

# Secret key used to run your flask app
secret_key = temporary_key

# Number of workers to run the Gunicorn web server
workers = 4

# The worker class gunicorn should use. Choices include
# sync (default), eventlet, gevent
worker_class = sync
like image 410
MishaVacic Avatar asked Jul 29 '17 10:07

MishaVacic


People also ask

Where is airflow PID?

The PID file for the webserver will be stored in $AIRFLOW_HOME/airflow-webserver. pid or in /run/airflow/webserver.

How do I check airflow service status?

To check the health status of your Airflow instance, you can simply access the endpoint /health . It will return a JSON object in which a high-level glance is provided. Please keep in mind that the HTTP response code of /health endpoint should not be used to determine the health status of the application.


2 Answers

For the sake of completeness: The question was answered elsewhere by Thomas:

Either stop the Java application that uses port 8080 already or change the port for Airflow.

Source

like image 88
Stev Avatar answered Oct 31 '22 07:10

Stev


The first time you run Airflow, it will create a file called airflow.cfg in your $AIRFLOW_HOME directory (~/airflow by default). This file contains Airflow’s configuration and you can edit it to change any of the settings.

You can find this file and search for web_server_port. Then change it to any unused port. You can then run airflow webserver as usual

like image 38
Ahmad Zareef Avatar answered Oct 31 '22 07:10

Ahmad Zareef