I'm very new to sql/hive. At first, I loaded a txt file into hive using:
drop table if exists Tran_data;
create table Tran_data(tran_time string,
resort string, settled double)
ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t' LINES TERMINATED BY '\n';
Load data local inpath 'C:\Users\me\Documents\transaction_data.txt' into table Tran_Data;
The variable tran_time
in the txt file is like this:10-APR-2014 15:01. After loading this Tran_data table, I tried to convert tran_time
to a "standard" format so that I can join this table to another table using tran_time
as the join
key. The date format desired is 'yyyymmdd'. I searched online resources, and found this: unix_timestamp(substr(tran_time,1,11),'dd-MMM-yyyy')
So essentially, I'm doing this: unix_timestamp('10-APR-2014','dd-MMM-yyyy')
. However, the output is "NULL".
So my question is: how to convert the date format to a "standard" format, and then further convert it to 'yyyymmdd' format?
In Hive, you are able to choose your preferred date format and calendar layout. To set your date preferences, navigate to your profile dropdown > 'My profile' > 'Edit profile'. 'Date Format' will allow you to change from US (MM/DD/YY) to international format (DD/MM/YY).
2.3 to_date(string timestamp) – Converts Timestamp string to Date type. to_date() function takes timestamp as an input string in the default format yyyy-MM-dd HH:mm:ss and converts into Date type.
Using "yyyyDDD" correctly converts Julian dates in Hive.
Hive supports two data types for Date/Time-related fields— Timestamp and Date : The Timestamp data type is used to represent a particular time with the date and time value. It supports variable-length encoding of the traditional UNIX timestamp with an optional nanosecond precision. It supports different conversions.
from_unixtime(unix_timestamp('20150101' ,'yyyyMMdd'), 'yyyy-MM-dd')
My current Hive Version: Hive 0.12.0-cdh5.1.5
I converted datetime in first column to date in second column using the below hive date functions. Hope this helps!
select inp_dt, from_unixtime(unix_timestamp(substr(inp_dt,0,11),'dd-MMM-yyyy')) as todateformat from table;
inp_dt todateformat
12-Mar-2015 07:24:55 2015-03-12 00:00:00
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With