Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

handling numeric null values using copy command in amazon redshift

while trying to load data to redshift using s3 I am getting an error

Invalid digit, Value 'N', Pos 0, Type: Decimal

its failing while trying to load NULL value to a numeric datatype column (column is nullable)

although i am using NULL AS '\000' conversion.

COPY campaign
FROM 's3://test/campaign.csv' 
credentials '------------------' 
EMPTYASNULL
NULL AS '\000' 
delimiter ',' 
region '----';

how can we handle such scenarios?

Table:

CREATE TABLE campaign ( 
  name VARCHAR(255) SORTKEY NOT NULL, 
  discount_med DECIMAL(5,2), 
  discount_packages DECIMAL(5,2), 
  discount_test DECIMAL(5,2) 
);

Sample input:

test1,5.25,NULL,1

like image 742
Kumar Pushkar Avatar asked Apr 11 '16 06:04

Kumar Pushkar


People also ask

How do you add null values in Redshift?

Simply use null , eg INSERT INTO t(a, b) SELECT 1, null; . Adding quotes around it makes it a text value.

Which COPY command would you use to load the data to Redshift?

The simplest COPY command uses the following format. COPY table-name FROM data-source authorization; The following example creates a table named CATDEMO, and then loads the table with sample data from a data file in Amazon S3 named category_pipe. txt .


1 Answers

Instead of NULL AS '\000' use NULL as 'NULL'. This worked for me on your table and data:

COPY campaign
FROM 's3://denis-stackoverflow/campaign.csv'
credentials '---'
NULL AS 'NULL'
EMPTYASNULL
delimiter ','
region 'ap-southeast-2';
like image 89
denismo Avatar answered Sep 18 '22 15:09

denismo