Airflow not loading dags in /usr/local/airflow/dags

Question

Airflow seems to be skipping the dags I added to /usr/local/airflow/dags.

When I run

airflow list_dags 

The output shows

[2017-08-06 17:03:47,220] {models.py:168} INFO - Filling up the DagBag from /usr/local/airflow/dags   ------------------------------------------------------------------- DAGS ------------------------------------------------------------------- example_bash_operator example_branch_dop_operator_v3 example_branch_operator example_http_operator example_passing_params_via_test_command example_python_operator example_short_circuit_operator example_skip_dag example_subdag_operator example_subdag_operator.section-1 example_subdag_operator.section-2 example_trigger_controller_dag example_trigger_target_dag example_xcom latest_only latest_only_with_trigger test_utils tutorial 

But this doesn't include the dags in /usr/local/airflow/dags

ls -la /usr/local/airflow/dags/ total 20 drwxr-xr-x 3 airflow airflow 4096 Aug  6 17:08 . drwxr-xr-x 4 airflow airflow 4096 Aug  6 16:57 .. -rw-r--r-- 1 airflow airflow 1645 Aug  6 17:03 custom_example_bash_operator.py drwxr-xr-x 2 airflow airflow 4096 Aug  6 17:08 __pycache__ 

Is there some other condition that neededs to be satisfied for airflow to identify a DAG and load it?

Answer 1

My dag is being loaded but I had the name of the DAG wrong. I was expecting the dag to be named by the file but the name is determined by the first argument to the DAG constructor

dag = DAG(     'tutorial', default_args=default_args, schedule_interval=timedelta(1)) 

Answer 2

Try airflow db init before listing the dags. This is because airflow list_dags lists down all the dags present in the database (And not in the folder you mentioned). Airflow initdb will create entry for these dags in the database.

Make sure you have environment variable AIRFLOW_HOME set to /usr/local/airflow. If this variable is not set, airflow looks for dags in the home airflow folder, which might not be existing in your case.