Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Importing CSV with commas in string values

I am trying to import a trivial CSV to Postgres 8.4 database:

Here is a table:

CREATE TABLE public.sample (
  a VARCHAR, 
  b VARCHAR
) WITHOUT OIDS;

Here is a CSV file sample:

"foo","bar, baz"

The query:

COPY sample FROM '/tmp/sample.csv' USING DELIMITERS ',';

Throws an exception:

ERROR:  extra data after last expected column
CONTEXT:  COPY sample, line 1: ""foo","bar, baz""

But, well, CSV parsing is not rocket science and I wonder if it is not possible to solve without reformatting the source CSV file.

Input comes from a 3rd party, I cannot change the format.
(I understand I could pre-process to change the delimiter before I import it.)

Final solution:

COPY sample FROM '/tmp/sample.csv' WITH DELIMITER ',' CSV;

Originally taken from https://stackoverflow.com/a/9682174/251311, and documentation page is http://www.postgresql.org/docs/8.4/static/sql-copy.html

like image 915
zerkms Avatar asked Apr 03 '12 21:04

zerkms


People also ask

How do you handle commas in data when importing a CSV file?

Re: Handling 'comma' in the data while writing to a CSV. So for data fields that contain a comma, you should just be able to wrap them in a double quote. Fields containing line breaks (CRLF), double quotes, and commas should be enclosed in double-quotes.

Can CSV files have commas?

A comma-separated values (CSV) file is a delimited text file that uses a comma to separate values. Each line of the file is a data record. Each record consists of one or more fields, separated by commas.

How do you import data with commas?

The best, quickest and easiest way to resolve the comma in data issue is to use Excel to save a comma separated file after having set Windows' list separator setting to something other than a comma (such as a pipe). This will then generate a pipe (or whatever) separated file for you that you can then import.


1 Answers

Clearly, you have a CSV file while you try to import it as text format. For Postgres 9.1 or newer, use:

COPY sample FROM '/tmp/sample.csv' (FORMAT csv);

The default delimiter for CSV format is the comma (,) anyway. More in the manual.
For PostgreSQL 8.4 or older:

COPY sample FROM '/tmp/sample.csv' CSV;
like image 148
Erwin Brandstetter Avatar answered Sep 27 '22 23:09

Erwin Brandstetter