Menu
In the image, there are 3 GenerateFlowFile processors connected to a LogAttribute processor. I would like to know the need for Funnel. Please clarify.
I understood one such use of funnel where it can be used to merge components from two different process group as described in this youtube video.
206
Data Provenance NiFi keeps a very granular level of detail about each piece of data that it ingests. As the data is processed through the system and is transformed, routed, split, aggregated, and distributed to other endpoints, this information is all stored within NiFi's Provenance Repository.
A NiFi Controller Service provides a shared starting point and functionality across Processors, other ControllerServices, and ReportingTasks within a single JVM. Controllers are used to provide shared resources, such as a database, ssl context, or a server connection to an external server and much more.
The Provenance Repository is where the history of each FlowFile is stored. This history is used to provide the Data Lineage (also known as the Chain of Custody) of each piece of data. Each time that an event occurs for a FlowFile (FlowFile is created, forked, cloned, modified, etc.) a new provenance event is created.
Input port is used to get data from the processor, which is not present in that process group. After dragging this icon, NiFi asks to enter the name of the Input port and then it is added to the NiFi canvas.
Well, imagine having 20 of those GenerateFlowFile processors. Imagine having to replace the LogAttribute processor with a different processor. It would be a hard job doing this since they are connected directly to the LogAttribute processor. However, if you had a Funnel between them, you would only need to replace the destination of the Funnel and not all of the processors.
I also use it for debugging flows. When I need a quick destination to test the intermediate result of a flow.
Basically what a Funnel does, is to just transfer FlowFiles forward as it gets them.
62
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
