Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Redshift COPY command with "^A" delimiter

I am trying to use a control A ("^A") delimited file to load into redshift using COPY command, I see default delimiter is pipe (|) and with CSV it is comma.

I couldnt file a way to use ^A, when i tried COPY command with ^A or \x01, it is throwing below message. Anybody tried this before? documentation says we can use delimiter, but no clue on using ^A.

Password: ERROR: COPY delimiter must be a single character

like image 506
cloudninja Avatar asked Mar 02 '16 13:03

cloudninja


People also ask

What does Copy command do in Redshift?

The COPY command appends the new input data to any existing rows in the table. The maximum size of a single input row from any source is 4 MB. To use the COPY command, you must have INSERT privilege for the Amazon Redshift table.

Which format is not supported by copy command?

Snowflake file format - ' ' is not recognized with Copy Into command.


1 Answers

I have used '\\001' as a delimiter for ctrl+A based field separation in redshift and also in Pig.

Example :

copy redshiftinfo from 's3://mybucket/data/redshiftinfo.txt' 
credentials 'aws_access_key_id=<access-key-id>;aws_secret_access_key=<secret-access-key>' 
delimiter '\\001' 
like image 64
rahulbmv Avatar answered Nov 03 '22 17:11

rahulbmv