Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Cannot use a "." in a Hive table column name

I'm using Hive 2.1.1 and I'm attempting to create a table with . in a column name:

CREATE TABLE `test_table`(
  `field.with.dots` string
);

When I do so I get:

FAILED: ParseException line 4:0 Failed to recognize predicate ')'. Failed rule: '[., :] can not be used in column name in create table statement.' in column specification

I must be doing something wrong because the hive documentation says:

In Hive release 0.13.0 and later, by default column names can be specified within backticks (`) and contain any Unicode character (HIVE-6013)

. is a unicode character. And idea what I might be doing?

To give you more context this is on an Amazon EMR 5.5.0 cluster. Thanks!

like image 366
Binary Logic Avatar asked May 05 '17 15:05

Binary Logic


1 Answers

Source code: HiveParser

...
private char [] excludedCharForColumnName = {'.', ':'};
...

  private CommonTree throwColumnNameException() throws RecognitionException {
    throw new FailedPredicateException(input, Arrays.toString(excludedCharForColumnName) + " can not be used in column name in create table statement.", "");
  }

Jira ticket :Disallow create table with dot/colon in column name

Please note the motivation:

Since we don't allow users to query column names with dot in the middle such as emp.no, don't allow users to create tables with such columns that cannot be queried

It seems create table was handled, but not CTAS nor ALTER TABLE...

hive> create table t as select 1 as `a.b.c`;
OK
hive> desc t;
OK
col_name    data_type   comment
a.b.c                   int                                         
Time taken: 0.441 seconds, Fetched: 1 row(s)
hive> select * from t;
FAILED: RuntimeException java.lang.RuntimeException: cannot find field a from [0:a.b.c]

hive> create table t (i int);
OK
hive> alter table t change column i `a.b.c` int
hive> select * from t;
Error while compiling statement: FAILED: RuntimeException java.lang.RuntimeException: cannot find field a from [0:a.b.c]

P.s.

I have updated the documentation (look for colon) https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL

like image 138
David דודו Markovitz Avatar answered Sep 30 '22 06:09

David דודו Markovitz