Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Variables and Parameters in Azure Data Factory?

Im just learning ADF, can you explain to me the difference between Variables and Parameters? I cant find a good explanation about it.

like image 525
Ruben Maldonado Avatar asked Nov 13 '18 13:11

Ruben Maldonado


People also ask

What is parameter and variable in Azure Data Factory?

Define a variable Select the "Variables" tab and click on "+ New" to define a new variable. Parameters can be of type String, Bool or Array. In this tab, you can also assign a default value to your variable that it will be used as initial value at the start of a pipeline run.

What are parameters in Azure Data Factory?

Parameter and expression concepts You can use parameters to pass external values into pipelines, datasets, linked services, and data flows. Once the parameter has been passed into the resource, it cannot be changed. By parameterizing resources, you can reuse them with different values each time.

What are system variables in Azure Data Factory?

These system variables can be referenced anywhere in the trigger JSON for triggers of type ScheduleTrigger. Time at which the trigger was scheduled to invoke the pipeline run. Time at which the trigger actually fired to invoke the pipeline run. This may differ slightly from the trigger's scheduled time.

How do I use variables in Azure Data Factory?

To use a Set Variable activity in a pipeline, complete the following steps: Select the background of the pipeline canvas and use the Variables tab to add a variable: Search for Set Variable in the pipeline Activities pane, and drag a Set Variable activity to the pipeline canvas.


2 Answers

Parameters are set for the whole pipeline, and are constant during a pipeline run. You can read them during a run to control what the pipeline does. They are like constants in a programming language, defined at the top of the source code.

Variables can be set at the start of a pipeline and read and modified during a run. They are like normal variables in a programming language.

like image 125
Chuck Connell Avatar answered Oct 26 '22 16:10

Chuck Connell


Parameters are something that can be set to model (dataset, pipeline, etc) at runtime. Without parameters, you model is fixed, for example, without parameters you can only write query like "select * from order where timestamp > '11/01/2018'" in dataset, with parameter 'TS' you can write query like "select * from order where timestamp > '@dataset.TS'" so that your activity run will query and copy data in different time window.

Variables are something that contain real value at runtime and can be assigned to those parameters in app model(dataset, pipeline, etc) to get real things done.

Here are some examples:

https://learn.microsoft.com/en-us/azure/data-factory/control-flow-expression-language-functions#examples

like image 35
Nicolas Zhang Avatar answered Oct 26 '22 16:10

Nicolas Zhang