Logo Questions Linux Laravel Mysql Ubuntu Git Menu
HTML CSS JAVASCRIPT SQL PYTHON PHP BOOTSTRAP JAVA JQUERY R React Kotlin
 

How to arrange Airflow Dags in the UI tidy in folders?

Tags:

I'm using Airflow version 1.9.0, and I'm gonna have hundreds of Dags. is there a way to arrange the Airflow UI with folders, sub folders, and only then to put the Dags in it?

enter image description here

like image 597
Maor Aharon Avatar asked Mar 28 '19 08:03

Maor Aharon

People also ask

How do I change DAGs folder in Airflow?

Change the Dag folder by using the following commands. Run the . bashrc file by using the below command. Now if you run airflow webserver, it will pick the dags from the AIRFLOW_HOME/dags directory.

Where are Airflow DAG files stored?

cfg . The default location for your DAGs is ~/airflow/dags .

How do I make a DAG in Airflow UI?

Step 1, define you biz model with user inputs Step 2, write in as dag file in python, the user input could be read by airflow variable model. (key/value mode) step 3. exchange tasks info by airflow xcom model. then it done.

What is the best view to spot any bottleneck in your DAG?

The graph view is perhaps the most comprehensive. Visualize your DAG's dependencies and their current status for a specific run.

2 Answers

The UI doesn't ship with that, but you can use the search bar to filter out what DAGs you're seeing. For instance you can type in your Owner name if you just want to see your own DAGs.

like image 173
chris.mclennon Avatar answered Oct 31 '22 21:10

chris.mclennon

My approach is make a consistent naming convention for dag_id, such as:

project1_workflow1_startdate1
project1_workflow2_startdate1
like image 32
Nguyen Van Duc Avatar answered Oct 31 '22 21:10

Nguyen Van Duc
Sign in to Comment

Related questions

pyqtdeploy: Unable to detect MSVC2015 or MSVC2017
Using webpack-merge to prepend loaders to a rules `use` array
Using List.of for immutable list with single element instead of Collections.singletonList
Merging Lists by Index in Python
Implementing Intersection over Union Loss Using Tensorflow
Bold row names of a formattable object
Rails app using docker-compose wont connect to the mysql container
How to solve the greatest-n-per-group problem with Entity Framework (Core)?
How to count groups of integers within arrays, without sorting the array?
How to avoid re-inserting data (duplicates) into SQL Server table while re-running SSIS package that loads data?
Rails 2.3 App problems running database migration
DataTables 1.10: Using saveState to remember filtering and order but need to update address URL to have shareable link for others

Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!