Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How can select a column and do a TRANSFORM in Hive?

Tags:

hive

I was using TRANSFORM USING with Hive 0.8.1, and noticed that this is invalid syntax:

SELECT
    a,
    TRANSFORM(b, c) USING 'fake.py' AS d,
FROM test_table;

Removing "a," makes this statement work. What is the correct way of using this?

like image 209
Enno Shioji Avatar asked May 01 '13 14:05

Enno Shioji


People also ask

What is transform in Hive?

When you use Create a Data Set in the Transformation Editor, your transformation script is applied to the source Hive table your project data set was created from. This operation creates a new Hive table in the Dgraph index and adds a new data set to the Catalog.

How do I SELECT specific rows in Hive?

hive SELECT Statement Select Specific Rows You can use the keyword RLIKE to use Java regular expressions. The following query will return rows which column name contains the words "smith" or "son". You can apply functions to the returned data. The following sentence will return all name in upper case.

Can we use like operator in Hive?

LIKE is an operator similar to LIKE in SQL. We use LIKE to search for string with similar text. RLIKE (Right-Like) is a special function in Hive where if any substring of A matches with B then it evaluates to true.


1 Answers

Apparently this is not possible. The fake.py has to handle that as well, i.e. one must do

SELECT
    TRANSFORM(a, b, c) USING 'fake.py' AS a, d
FROM test_table;

and make it so that fake.py does output 'a' as well.

like image 69
Enno Shioji Avatar answered Sep 28 '22 03:09

Enno Shioji