Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Map type variable in hive

Tags:

hive

hiveql

I am having trouble trying to define map type in hive. According to Hive Manual there definitely is a map type, unfortunately there aren't any examples on how to use it. :-(

Suppose, I have a table (users) with following columns:

Name     Ph    CategoryName

This "CategoryName" column has specific set of values. Now I want to create a hashtable that maps CategoryName to CategoryID. I tried doing:

set hivevar:nameToID=map('A',1,'B',2); 

I have 2 questions:

  1. When I do set hivevar:${nameToID['A']} I thought that it would print out value as 1. But I get "${hivevar:nameToID['A']} is undefined"

  2. I am not sure how can I say something like, select name, ph, ${nameToID[CategoryName]} from users

like image 458
test123 Avatar asked Jan 25 '13 03:01

test123


People also ask

What is map data type in Hive?

map: It is an unordered collection of key-value pairs. Keys must be of primitive types. Values can be of any type. struct: It is a collection of elements of different types. Examples: complex datatypes.

How do I change datatype in Hive?

By using this command below one can change the column data type: ALTER TABLE table_name CHANGE column_name column_name new_datatype; I hope this works.

What is struct data type in Hive?

STRUCT : The struct data type in Hive is analogous to the STRUCT in C programming language. It is a record type that holds a set of named fields that can be of any primitive data types. Fields in the STRUCT type are accessed using the DOT ( . ) notation.

Does Hive support varchar data type?

String Types It contains two data types: VARCHAR and CHAR. Hive follows C-types escape characters.


1 Answers

Let's assume you have the following table:

describe test;
name      string    
ph        string    
category  map<string,int>

select * from test;
name    ph  category
Name1   ph1 {"type":1000,"color":200,"shape":610}
Name2   ph2 {"type":2000,"color":200,"shape":150}
Name3   ph3 {"type":3000,"color":700,"shape":167}

Accessing the map column :

select ph, category["type"], category["color"] from test;
ph1    1000    200
ph2    2000    200
ph3    3000    700

An equivalent using a Hive variable:

set hivevar:nameToID=
   map("t", category["type"], "c", category["color"], "s", category["shape"]);

select ph, ${nameToID}["t"], ${nameToID}["c"] from test;
ph1    1000    200
ph2    2000    200
ph3    3000    700

This works on Hive 0.9.0

like image 175
Lorand Bendig Avatar answered Oct 14 '22 12:10

Lorand Bendig