How do I raise an error from if activity in ADFV2?

Question

I have a child pipeline that receives instructions from a control file using a dataset. These instructions define what directory to copy files from.

First this child pipeline checks that a file exists via Get Metadata activity on the source folder. It then executes a child pipeline to process the data if one or more childitems are returned from GetMetaData.

In the control dataset, there is also a required Y/N field meaning I can ignore the error if the folder or file does not exist.

If the folder does not exist, the GetMEtadata will fail. If it exists but no files, I get 0 child items. So 2 different things happen for file or folder missing (an error or 0 items).

In either case, I need to route the output of GetMetaData to an IF that checks if the file was required. If not, consume the error and return. If it was required, raise an error.

I can't find a way to raise an error though. Just as important, is there an alternative approach that would work or fit better with ADF V2 design?

Many thanks, Mark.

Answer 1

In November 2021 ADF gained the Fail activity, in the General group.

It has two specific properties, Fail message and Error code, both of which are dynamic. They behave exactly as would be expected.

When executed it always has the status Failed. The message and code are evaluated and appear in the pipeline output.

Answer 2

I have found the easiest way to throw an error is to do a SQL lookup on the following query:

THROW 99999, 'Error message thrown', 1

For us, we make extensive use of Azure Function Apps, so we created a special function called "ThrowError" that throws an error with a message provided as a parameter. That is the easiest only if you already use Azure Function Apps, not worth it to just create the one function.