Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

fread unable to read .csv files with first column empty

Tags:

r

csv

data.table

Say I have the first test.csv that looks like this

,a,b,c,d,e

If I try to read it using read.csv, it works fine.

read.csv("test.csv",header=FALSE)
#  V1 V2 V3 V4 V5 V6
#1 NA  a  b  c  d  e
#Warning message:
#In read.table(file = file, header = header, sep = sep, quote = quote,  :
#  incomplete final line found by readTableHeader on 'test.csv'

However, if I attempt to read this file using fread, i get an error instead.

require(data.table)
fread("test.csv",header=FALSE)
#Error in fread("test.csv", header = FALSE) : 
#  Not positioned correctly after testing format of header row. ch=','

Why does this happen and what can I do to correct this?

like image 904
Wet Feet Avatar asked Mar 12 '14 06:03

Wet Feet


1 Answers

As for me, my problem was only that the first ? rows of my file had a missing ID value.

So I was able to solve the problem by specifying autostart to be sufficiently far into the file that a nonmissing value popped up:

fread("test.csv", autostart = 100L, skip = "A")

This guarantees that when fread attempts to automatically identify sep and sep2, it does so at a well-formatted place in the file.

Specifying skip also makes sure fread finds the correct row in which to base the names of the columns.

If indeed there are no nonmissing values for the first field, you're better off just deleting that field from the .csv with Richard Scriven's approach or a find-and-replace in your favorite text editor.

like image 169
MichaelChirico Avatar answered Sep 29 '22 12:09

MichaelChirico