Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

PostgreSQL copy from CSV with missing data values

Tags:

I'm trying to import a CSV file into PostgreSQL using COPY. It chokes when it hits a row where there are empty values, e.g. the second row below:

JAN-01-2001,1,2,3,4,5

JAN-02-2001,6,7,,,

I've tried this COPY statement, as well as variants using NULL and QUOTE and havent found anything that works.

COPY data FROM 'data.dat' USING DELIMITERS ',' CSV;

Any suggestions? The data file is in a massive 22GB flat file, so I'd like to avoid editing it directly.

like image 929
ugh Avatar asked Dec 01 '11 19:12

ugh


People also ask

Does psql COPY overwrite?

If you COPY TO a file already containing data, the existing data will be overwritten.

How do I COPY a CSV file into a Postgres table?

After creating the sample CSV file and table, you can now easily execute the PostgreSQL Import CSV task via any of the following methods: Method 1: Perform PostgreSQL Import CSV Job using the COPY Command. Method 2: Perform PostgreSQL Import CSV Job using PgAdmin.

What is psql Copy command?

Psql \copy command is used when you want to export the data from Postgres table to a CSV file on a client machine. To use this command, you will need access to the psql prompt. You will understand it more with the following psql copy examples. To copy the entire table to a csv file, use \copy.

How do you COPY data from one database to another database in Postgres?

PostgreSQL copy database within the same server: PostgreSQL makes it simple to do so using the CREATE DATABASE statement as follows: Syntax: CREATE DATABASE target_database WITH TEMPLATE source_database; This statement copies the source_database to the target_database.


1 Answers

I would suggest converting your numeric columns to text columns for the purposes of your import. The reason is that an empty string is not a valid numeric value. Change your numeric columns to text columns, import the CSV file, update the empty values to null or 0, and then change the column back to an integer.

like image 94
Kenaniah Avatar answered Sep 19 '22 19:09

Kenaniah