Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Convert all fields ending with "id" to integer using convert in mutate?

Currently I am doing something like this in my logstash config file :

filter {
    ...
    mutate {
        ...
        convert => {
            "blahId" => "integer"
            "blahblahId" => "integer"
            ...
            ...
            "b...blahId" => "integer"
        }
        ...
    }
    ...
}

So basically I want to convert all the fields ending with "Id" to type integer. Is there a way to do that in one line? Something like "*Id" => "integer" which does that ?

Edit : I tried

convert => {
    "*Id" => "integer"
}

As I expected, didn't work.

Using ruby filter perhaps ?

like image 208
Karup Avatar asked Jul 04 '16 06:07

Karup


1 Answers

Not specifically an answer to this but this is what I end up doing :

ruby {
        code => "
            fieldArray = event['kvmess'].split('|');
            for field in fieldArray
                name = field.split('=')[0];
                value = field.split('=')[1];
                if value =~ /\A\d+\Z/
                    event[name] = value.to_i
                else
                    event[name] = value
                end
            end
        "
    }

My kvmess was like "blahId=123|blahblahId=456|some=thing|b..blahId=789". So this converted all keys having numeric values to type integer.

There is a plugin - kv meant specifically for this but it does not have the functionality to change datatype to int so I ended up using this ruby plugin instead.

like image 139
Karup Avatar answered Nov 09 '22 14:11

Karup