For processing logs I want to use Apache Hive regEx serde but I only found examples that use String as datatype for the columns of the table.
Now my question is: are datebased types and integers and arrays supported or is it just strings?
This example (and others) only uses strings:
CREATE TABLE access_log ( remote_ip STRING, request_date STRING, method STRING, request STRING, protocol STRING ) ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.RegexSerDe' WITH SERDEPROPERTIES ( "input.regex" = "([^ ]) . . [([^]]+)] \"([^ ]) ([^ ]) ([^ \"])\" *", "output.format.string" = "%1$s %2$s %3$s %4$s %5$s" ) STORED AS TEXTFILE
hope this helps :)
I haven't used the RegexSerDe personally, but I do notice that there are two classes for it: serde/src/java/org/apache/hadoop/hive/serde2/RegexSerDe.java contrib/src/java/org/apache/hadoop/hive/contrib/serde2/RegexSerDe.java
The second one, which you are referring to, does indeed appear to be restricted to strings. The other appears to support primitive types.
For whatever reason I only see the second one referenced in the API docs.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!Donate Us With