How to compare NiFi file content?

Question

I'm new to NiFi. I was asked to fetch file from FTP server. If the file content is same as previous one, then do nothing, otherwise send the new file to another FTP server. Any ideas ?

Thanks,

Richard

Answer 1

The HashContent processor and DetectDuplicate processor can be used for this.

HashContent will create a hash of the flow file content and put it in an attribute called hash.value.

DetectDuplicate will check a cache of hashed values to see if the same hash exists and then route it to the appropriate relationship (duplicate or non-duplicate).

https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.8.0/org.apache.nifi.processors.standard.DetectDuplicate/index.html

https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.8.0/org.apache.nifi.processors.standard.HashContent/index.html