Refreshing dags without web server restart apache airflow

Question

Is there any way to reload the jobs without having to restart the server?

Answer 1

In your airflow.cfg, you've these two configurations to control this behavior:

# after how much time a new DAGs should be picked up from the filesystem
min_file_process_interval = 0
dag_dir_list_interval = 60

You might have to reload the web-server, scheduler and workers for your new configuration to take effect.

Answer 2

I had the same question, and didn't see this answer yet. I was able to do it from the command line with the following:

python -c "from airflow.models import DagBag; d = DagBag();"

When the webserver is running, it refreshes dags every 30 seconds or so by default, but this will refresh them in between if necessary.

Answer 3

Dags should be reloaded when you update the associated python file.

If they are not, first try to manually refresh them in UI by clicking the button that looks like a recycle symbol:

If that doesn't work, delete all the .pyc files in the dags folder.

Usually though, when I save the python file the dag gets updated within a few moments.