Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Is it possible to import a CSV file to an existing table without the headers being included?

Tags:

import

sqlite

csv

I'm trying to import a CSV file to a table that is empty but already exists in an SQLite database. For example:

sqlite> CREATE TABLE data (...);
sqlite> .mode csv
sqlite> .import mydata.csv data

I have created the table in advance because I'd like to specify a primary key, data types, and foreign key constraints. This process works as expected, but it unfortunately includes the header row from the CSV file in the table.

Here's what I've learned from the SQLite docs regarding CSV imports:

There are two cases to consider: (1) Table "tab1" does not previously exist and (2) table "tab1" does already exist.

In the first case, when the table does not previously exist, the table is automatically created and the content of the first row of the input CSV file is used to determine the name of all the columns in the table. In other words, if the table does not previously exist, the first row of the CSV file is interpreted to be column names and the actual data starts on the second row of the CSV file.

For the second case, when the table already exists, every row of the CSV file, including the first row, is assumed to be actual content. If the CSV file contains an initial row of column labels, that row will be read as data and inserted into the table. To avoid this, make sure that table does not previously exist.

So basically, I get extra data because I've created the table in advance. Is there a flag to change this behavior? If not, what's the best workaround?

like image 295
jncraton Avatar asked Oct 27 '17 19:10

jncraton


People also ask

Do CSV files always have headers?

The CSV format is not standardized, so various implementations exist. In 2005, the Internet Society published guidelines for creating CSV files. They wrote down best practices to structure and process CSV data. From those guidelines and giving the lack of standardization, the header line is optional in a CSV file.

How do I read a CSV file without headers?

To read CSV file without header, use the header parameter and set it to “None” in the read_csv() method.

How do I save a CSV file without the header?

pandas to CSV without Header To write DataFrame to CSV without column header (remove column names) use header=False param on to_csv() method.


1 Answers

The sqlite3 command-line shell has no such flag.

If you have a sufficiently advanced OS, you can use an external tool to split off the first line:

sqlite> .import "|tail -n +2 mydata.csv" data
like image 157
CL. Avatar answered Sep 29 '22 15:09

CL.