Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

What file format do we need to upload variables through the admin in airflow?

I'm trying to define a variable file to upload through the airflow UI and I'm struggling to find the correct format to upload.

I've created a file called settings.cfg that looks like:

my_var=test

However when I try to upload this through the Airflow Admin->Variables dialog I get the following message:

Missing file or syntax error.

What am I missing?

like image 945
Bovard Avatar asked Nov 14 '17 18:11

Bovard


People also ask

How do you add a variable in Airflow?

Airflow Variables can also be created and managed using Environment Variables. The environment variable naming convention is AIRFLOW_VAR_{VARIABLE_NAME} , all uppercase. So if your variable key is FOO then the variable name should be AIRFLOW_VAR_FOO .

How do you pass a variable in Airflow?

You can pass parameters from the CLI using --conf '{"key":"value"}' and then use it in the DAG file as "{{ dag_run. conf["key"] }}" in templated field. Save this answer.

Where are Airflow variables stored?

It is important that you restrict the number of Airflow variables in your DAG because variables are stored inside the Metadata database so every single time when the variable is used, connection to the database is established.


1 Answers

It expects JSON.

{
  "my_var": "test",
  "your_var": "hello"
}

Reference: https://github.com/apache/incubator-airflow/blob/1.8.2/airflow/www/views.py#L1766-L1779

like image 139
Daniel Huang Avatar answered Oct 05 '22 19:10

Daniel Huang