Math functions in Logstash

Question

I am looking forward to use mathematical operations on the input received in Logstash, but unable to see any of such filter.

Input is as following:

{
  "user_id": "User123",
  "date": "2016 Jun 26 12:00:12",
  "data": {
    "doc_name": "mydocs.xls",
    "doc_size": "8526587",
  }
}

The "doc_size" field will have bytes, I would like to add a new field say "doc_size_mb" which will contain the size in MB's.

So I want a simple division operation here like:

doc_size_mb = doc_size/(1024*1024)

I could see a link which says Logstash has math filter, but this is not visible here .

Answer 1

The logstash-filter-math is not a core plugin but it is available here. You can follow the next steps in order or install it:

> git clone https://github.com/robin13/logstash-filter-math.git
> cd logstash-filter-math
> gem build
> $LS_HOME/bin/logstash-plugin install logstash-filter-math-0.2.gem

If you don't want to install a 3rd party plugin just for that, you can also easily achieve the same computation with a ruby filter:

filter {
   ruby {
      code => "event['data']['doc_size_mb'] = event['data']['doc_size'].to_i / (1024 * 1024)"
   }
}

Answer 2

I tried using the above approach to multiply an existing field by a factor value and update the value of the existing field in the event by this new scaled value in Logstash 7.0.1, but it did not work as expected. I modified it to use the Event API's set() and get() methods which worked out for me.

Initial approach (did not work) -

filter {
     ruby {
        code => "event['data']['myField'] = event['data']['myField'].to_i * 0.25"
          }
       }

Working solution -

filter {
     ruby {
        code => "event.set('myField',event.get('myField')* 0.25)
          }
       }