Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Hive modify partitioned table data

Tags:

hadoop

hive

Problem: One column value is null. It should be 'ab'. Unfortunately I have written '' instead 'ab'.

My table is partitioned table. Is there any way to change that?

I found the following way. But it seems inefficient.

  1. Create a temp table like my table
  2. Use INSERT OVERWRITE. Read data from my old table and write to new table. I am using case statement to change '' to 'ab'
  3. And then change my temp table to original table.

I am looking for a solution something like update partition and msck. Is there any way to do?

like image 440
Gibbs Avatar asked Feb 19 '26 18:02

Gibbs


1 Answers

You can overwrite single partition in this way:

set hive.exec.dynamic.partition=true;
set hive.exec.dynamic.partition.mode=nonstrict;

insert overwrite target_table partition (part_col)
select 
case when column ='' then 'ab' else column end as column ,
col2,    --select all the columns in the same order
col3,
part_col --partition column is the last one
from target_table where part_col='your_partition_value';
like image 91
leftjoin Avatar answered Feb 21 '26 14:02

leftjoin



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!