Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

COPY command: copy only specific columns from csv

I had a question surrounding the COPY command in PostgreSQL. I have a CSV file that I only want to copy some of the columns values into my PostgreSQL table.

Is it possible to do this? I am familiar with using the COPY command to copy all of the data from a CSV into a table using the header to map to the column names but how is this possible when I only want some of the columns?

like image 256
parchambeau Avatar asked Mar 29 '13 16:03

parchambeau


People also ask

How do I extract a column from a csv file?

Make a list of columns that have to be extracted. Use read_csv() method to extract the csv file into data frame. Print the exracted data. Plot the data frame using plot() method.

How do I select an entire column in CSV?

To select an entire column, click the column letter or press Ctrl+spacebar.

What is true for COPY command in PostgreSQL?

The COPY command moves data between PostgreSQL tables and standard file system files. COPY TO copies the contents of the table to the file. COPY TO can also copy the results of the SELECT query. That is, if the column list is specified, COPY TO only copies the data in the specified columns to the file.

What separates columns in CSV?

CSV is a delimited data format that has fields/columns separated by the comma character and records/rows terminated by newlines.


1 Answers

Either pre-process the CSV file, or (what I probably would do) import into a temporary copy of the target table and INSERT only selected columns in a second step:

CREATE TEMP TABLE tmp AS SELECT * FROM target_table LIMIT 0;
ALTER TABLE tmp ADD COLUMN etra_column1 text
             ,  ADD COLUMN etra_column2 text;  -- add excess columns
COPY tmp FROM '/path/tp/file.csv';

INSERT INTO target_table (col1, col2, col3)
SELECT col1, col2, col3 FROM tmp  -- only reelvant columns
WHERE  ...  -- optional, to also filter rows

A temporary table is dropped automatically at the end of the session. If the processing takes longer, use a regular table.

like image 63
Erwin Brandstetter Avatar answered Sep 30 '22 18:09

Erwin Brandstetter