Menu
Airflow is returning an error when trying to run a DAG saying that it can't find an environment variable, which is odd because it's able to find 3 other environment variables that I'm storing as a Python variable. No issues with those variables at all.
I have all 4 variables in ~/.profile and have also done
export var1="varirable1"
export var2="varirable2"
export var3="varirable3"
export var4="varirable4"
Under what user does airflow run? I've done those export commands under sudo as well, so I thought they would be picked up by airflow when it runs the dag
562
Airflow uses Fernet to encrypt variables stored in the metastore database. It guarantees that without the encryption password, content cannot be manipulated or read without the key.
Variables in Airflow are a generic way to store and retrieve arbitrary content or settings as a simple key-value store within Airflow. Variables can be listed, created, updated, and deleted from the UI (Admin -> Variables), code, or CLI. In addition, JSON settings files can be bulk uploaded through the UI.
On the Windows taskbar, right-click the Windows icon and select System. In the Settings window, under Related Settings, click Advanced system settings. On the Advanced tab, click Environment Variables. Click New to create a new environment variable.
Is it maybe because airflow uses non-login shell? Have you tried putting these lines in : ~/.bashrc instead of ~/.profile ?
137
As per this answer, the variables should be put in /etc/default/airflow (on Debian/Ubuntu) or /etc/sysconfig/airflow (on Centos/Redhat).
31
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
