Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to point to the airflow unittest.cfg?

Tags:

airflow

Airflow creates a unittest.cfg file in the AIRFLOW_HOME environment variable path.

My question is: how can I point to unittest.cfg in the same why that I point to the airflow.cfg via the environment variable AIRFLOW_CONFIG?

The reason why I want to do this is because I don't want to have any config files in the AIRFLOW_HOME directory.

Also, if anyone knows better, could you please explain what is the unittest.cfg is for as there is no documentation I could find on it.

like image 951
Newskooler Avatar asked Mar 08 '19 15:03

Newskooler


People also ask

Where do I put airflow cfg?

The first time you run Airflow, it will create a file called airflow. cfg in your $AIRFLOW_HOME directory ( ~/airflow by default). This file contains Airflow's configuration and you can edit it to change any of the settings.

How do you test an airflow operator?

DAG validation tests apply to all DAGs in your Apache Airflow environment, so only one test suite is required. Simply run the Python file to see if your DAG can be loaded, indicating that there are no syntax errors.


1 Answers

unittest.cfg test configuration file is the default configuration file used when Airflow is running in test mode.

Test mode can be activated by setting the unit_test_mode configuration option in airflow.cfg or AIRFLOW__CORE__UNIT_TEST_MODE environment variable to True .

The configuration values in test configuration file overwrite those in airflow.cfg in runtime when test mode is activated.

# Source: https://github.com/apache/airflow/blob/1.10.5/airflow/configuration.py#L558,L561
def get_airflow_test_config(airflow_home):
    if 'AIRFLOW_TEST_CONFIG' not in os.environ:
        return os.path.join(airflow_home, 'unittests.cfg')
    return expand_env_var(os.environ['AIRFLOW_TEST_CONFIG'])

The AIRFLOW_TEST_CONFIG environment variable can be set to the path of your test configuration file.

like image 50
Oluwafemi Sule Avatar answered Sep 28 '22 08:09

Oluwafemi Sule