Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Date Format Conversion in Hive

Tags:

date

format

hive

I'm very new to sql/hive. At first, I loaded a txt file into hive using:

drop table if exists Tran_data;
create table Tran_data(tran_time string, 
resort string, settled double)
ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t' LINES TERMINATED BY '\n';
Load data local inpath 'C:\Users\me\Documents\transaction_data.txt' into table Tran_Data;

The variable tran_time in the txt file is like this:10-APR-2014 15:01. After loading this Tran_data table, I tried to convert tran_time to a "standard" format so that I can join this table to another table using tran_time as the join key. The date format desired is 'yyyymmdd'. I searched online resources, and found this: unix_timestamp(substr(tran_time,1,11),'dd-MMM-yyyy')

So essentially, I'm doing this: unix_timestamp('10-APR-2014','dd-MMM-yyyy'). However, the output is "NULL".

So my question is: how to convert the date format to a "standard" format, and then further convert it to 'yyyymmdd' format?

like image 625
Yuning Zhang Avatar asked Aug 07 '14 18:08

Yuning Zhang


People also ask

How do I change the date format in Hive?

In Hive, you are able to choose your preferred date format and calendar layout. To set your date preferences, navigate to your profile dropdown > 'My profile' > 'Edit profile'. 'Date Format' will allow you to change from US (MM/DD/YY) to international format (DD/MM/YY).

How do I convert Yyyymmdd to date in Hive?

2.3 to_date(string timestamp) – Converts Timestamp string to Date type. to_date() function takes timestamp as an input string in the default format yyyy-MM-dd HH:mm:ss and converts into Date type.

How do you convert Julian date to normal date in Hive?

Using "yyyyDDD" correctly converts Julian dates in Hive.

What is the data type for date in Hive?

Hive supports two data types for Date/Time-related fields— Timestamp and Date : The Timestamp data type is used to represent a particular time with the date and time value. It supports variable-length encoding of the traditional UNIX timestamp with an optional nanosecond precision. It supports different conversions.


2 Answers

from_unixtime(unix_timestamp('20150101' ,'yyyyMMdd'), 'yyyy-MM-dd') 
like image 139
jimbo Avatar answered Oct 20 '22 00:10

jimbo


My current Hive Version: Hive 0.12.0-cdh5.1.5

I converted datetime in first column to date in second column using the below hive date functions. Hope this helps!

select inp_dt, from_unixtime(unix_timestamp(substr(inp_dt,0,11),'dd-MMM-yyyy')) as todateformat from table;

inp_dt todateformat
12-Mar-2015 07:24:55 2015-03-12 00:00:00

like image 3
manojd7sto Avatar answered Oct 19 '22 22:10

manojd7sto