I have a column in my dataset that has a datatype of bigint:
Col1 Col2
1 1519778444938790
2 1520563808877450
3 1519880608427160
4 1520319586578960
5 1519999133096120
How do I convert Col2 to the following format:
year-month-day hr:mm:ss
I am not sure what format my current column is in but I know that it is supposed to be a timestamp.
Any help will be great, thanks!
You can convert timestamp to date with cast(col as date) or date(col) .
You can use from_unixtime/to_timestamp function in spark to convert Bigint column to timestamp .
You can simply use the fromtimestamp function from the DateTime module to get a date from a UNIX timestamp. This function takes the timestamp as input and returns the corresponding DateTime object to timestamp.
Have you tried to use functions like from_unixtime
? You could use it to convert unix time to timestamp, then you could use date_format
to display it in way you want. Notice that in your example your unix time is with microseconds, so you might want to convert it first to milliseconds.
I have not tested that but I am assuming that your code should look like:
date_format(from_unixtime(col2/1000), '%Y-%m-%d %h:%i:%s')
Notice that from_unixtime
accepts also a time zone.
Please visit this page to see the more details about date related functions: https://docs.starburstdata.com/latest/functions/datetime.html
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With