Importing local module (python script) in Airflow DAG

Question

I'm trying to import a local module (a python script) to my DAG.

Directory structure:

airflow/ ├── dag │   ├── __init__.py │   └── my_DAG.py └── script     └── subfolder         ├── __init__.py         └── local_module.py  

Sample code in my_DAG.py:

#trying to import from local module from script.subfolder import local_module    #calling a function in local_module.py   a = some_function()   

I get an error in Airflow saying 'Broken DAG: my_DAG. No module named 'local_module'.

I've updated Airflow to 1.9.0 but this doesn't fix the issue.

• What is the solution here?
• I also read somewhere that I could solve this by creating a plugin. Can anyone point to how I can do this?

Thanks.

Answer 1

This usually has to do with how Airflow is configured.

In airflow.cfg, make sure the path in airflow_home is correctly set to the path the Airflow directory strucure is in.

Then Airflow scans all subfolders and populates them so that modules can be found.

Otherwise, just make sure the folder you are trying to import is in the Python path: How to use PYTHONPATH