Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

"Handling signal: ttou" message while running DAG in airflow

I have created sample DAG, where I had DAG config as below.

default_args = {
    'owner': 'airflow',         
    'depends_on_past': False, 
    'start_date': one_min_ago,
    'email': ['[email protected]'],
    'email_on_failure': True, 
    'email_on_retry': True, 
    'retries': 5, 
    'retry_delay': timedelta(hours=30))

With this when I run airflow webserver I'm getting below message.

/home/af_user/anaconda/lib/python3.5/site-packages/flask/exthook.py:71: 
ExtDeprecationWarning: Importing flask.ext.cache is deprecated, use 
flask_cache instead.
 .format(x=modname), ExtDeprecationWarning
[2017-12-18 12:41:27,967] [17328] {models.py:167} INFO - Filling up the 
DagBag from /home/af_user/airflow/dags
[2017-12-18 12:41:28 +0000] [16648] [INFO] Handling signal: ttou
[2017-12-18 12:41:57 +0000] [16655] [INFO] Worker exiting (pid: 16655)

And also DAG will be there in running state only.

Let me know if any once came across this issue and fixed it already.

like image 384
Somashekar Muniyappa Avatar asked Dec 18 '17 12:12

Somashekar Muniyappa


2 Answers

Those messages are expected. The ttou (and ttin) signals are used to refresh gunicorn workers of the webserver so that it picks up DAG changes. You can modify or disable this behavior with the worker_refresh_interval and worker_refresh_batch_size airflow config values.

like image 70
Daniel Huang Avatar answered Nov 13 '22 04:11

Daniel Huang


I don't really want to change worker_refresh_interval or worker_refresh_batch_size without a good reason. An alternative is to set an environment variable:

GUNICORN_CMD_ARGS="--log-level WARNING"

If setting this in a docker-compose.yml file, the following is tested with apache-airflow==1.10.6 with gunicorn==19.9.0:

environment:
    - 'GUNICORN_CMD_ARGS=--log-level WARNING'

If setting this in a Dockerfile, the following is tested with apache-airflow==1.10.6 with gunicorn==19.9.0:

ENV GUNICORN_CMD_ARGS --log-level WARNING

Credit: answer by amoskaliov

like image 30
Asclepius Avatar answered Nov 13 '22 04:11

Asclepius