Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Filter Array in Hive

An Apache hive table has the following column definition:

myvars:array<struct<index:bigint,value:string>>

An example for the corresponding data is:

"myvars":[
  {"index":2,"value":"value1"}
  , {"index":1,"value":"value2"}
  , {"index":2,"value":"value3"}
]

How can this array be filtered to all elements where "index"==2.

In JavaScript I would do something like the following:

myvars.filter(function(d){return d.index==2;})

How can the same result be achieved with Apache Hive QL, preferably without lateral views?

like image 261
user1091141 Avatar asked Oct 30 '22 17:10

user1091141


1 Answers

In hive you have a set of Collection functions:

 Collection
    array_contains(Array<T> a, val)
    array<K.V> map_keys(Map<K.V> a)
    array<K.V> map_values(Map<K.V> a)
    size(Map<K.V>|Array<T> a)
    sort_array(Array<T> a)

in your query use

...
WHERE
array_contains(myvars,2) 
like image 63
Alg_D Avatar answered Nov 15 '22 08:11

Alg_D