Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to get all values with expression language in Apache Nifi Process Property

I am trying to create an Apache Nifi model that allows me to read in all my data from Elasticsearch and store it to a file. I have everything connected correctly but the issue I am having is the FetchElasticsearch process is requiring a document identifier (as it should) but I want to get every single item in the index it is searching not just a document with ID 1 for example. I know that Nifi and the process property supports expression language so I have tried simply using a regex expression that should match all characters which should just be

${'*'}

but I get warning when I did this because the process actually looks for the literal document id that is * which of course does not exist. Below are screenshots so hopefully it can help with understanding my issue.

Nifi Flow

FetchElasticsearch Process Property

I am searching localhost:9300/tweet_library/tweet/(regex expression) so I want all of the documents in tweet_library. Any help is appreciated, thanks.

like image 591
Troy Zuroske Avatar asked Nov 28 '25 08:11

Troy Zuroske


1 Answers

Answer from horton works community:

"The FetchElasticsearch processor uses the Get API, which requires a single document identifier and doesn't support regular expressions. As an alternative, you can use InvokeHttp to call the Multi-Get API or the Search API, which give you more control over the retrieval of multiple documents."

like image 90
Troy Zuroske Avatar answered Nov 29 '25 22:11

Troy Zuroske



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!