Menu
I'm using ADFv2 and I would like to use some optional parameters defined at the dataset level. I didn't found if that can be achieved.
If I define a parameter in the dataset (after defining a parameter at the pipeline level and connecting the two so the pipeline passes the value to the parameter at the dataset level) when I run the pipeline manually and don't specify a value I get the error
No value provided for Parameter 'parameter'
926
To add parameters to your data flow, click on the blank portion of the data flow canvas to see the general properties. In the settings pane, you will see a tab called Parameter. Select New to generate a new parameter. For each parameter, you must assign a name, select a type, and optionally set a default value.
Let's compare Azure Data Factory Version 1 and Version 2 at a high level. ADFv1 – is a service designed for the batch data processing of time series data. ADFv2 – is a very general-purpose hybrid data integration service with very flexible execution patterns.
To create a global parameter, go to the Global parameters tab in the Manage section. Select New to open the creation side-nav. In the side-nav, enter a name, select a data type, and specify the value of your parameter. After a global parameter is created, you can edit it by clicking the parameter's name.
The standard and only way possible is to parameterize a linked service and pass dynamic values at run time by defining the values in ADF. For example, if you want to connect to different databases on the same logical SQL server, you can now parameterize the database name in the linked service definition.
Just for anyone that may be looking to do something like this. At least until today's date we can't have optional parameters for a pipeline, this is by design according to what the Product Team of ADF has told me through a support ticket in Azure.
196
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
