Setting custom environment variables in managed apache airflow

Question

We are planning to switch from managing airflow ourselves to Managed Apache Airflow services of AWS. Our original dags use some custom environment variables that need to be set in Managed airflow as well. So far I was not able to find a way to set custom environment variables while setting up airflow environment in MWAA. Please let me know if anyone knows how to set them.

Answer 1

An approach for setting environment variables is to use Airflow Variables.

In MWAA, you can store Airflow Variables in AWS Secrets Manager. This approach is documented in MWAA's official documentation. Note that this approach requires specific configuration for the MWAA environment. Once the MWAA environment is set up and the variables are stored in AWS Secrets Manager, the variables become accessible through the Airflow Variable APIs.

from airflow.models import Variable

# Normal call style
foo = Variable.get("foo")

See Step two: Create the Secrets Manager backend as an Apache Airflow configuration option
See Step four: Add the variables in Secrets Manager