Extract Date from epoch in NiFi

Question

I have a CSV file with an attribute having epoch values like '1517334599.906'.

I want to convert/update the Epoch values into ISO timestamp 'yyyy-MM-dd HH:mm:ss.SSS' via NiFi.

That conversion is for Kibana to recognize the field as Timestamp. Is there a way to do this? If there is can anyone help me with the configuration?

Answer 1

Using NiFi's record capabilities you can use UpdateRecord with a CsvReader and CsvWriter.

See the "format" function in expression language for converting an epoch to a date string:

https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#format

In UpdateRecord you would do something like:

/eventDate = ${field.value:format("yyyy-MM-dd HH:mm:ss.SSS")} 

This says take the value of /eventDate (change this to your field name) and set the value of that field to the result of the format function on the right.

The only thing I am not sure about is whether an epoch can have a decimal portion as shown in your example. I would expect it to be converted to a long which would be a whole number.