Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

'Incomplete final line' warning when trying to read a .csv file into R

People also ask

What does incomplete final line mean in R?

The “incomplete final line” error message arises when there is a “missing” return in the last data row of your CSV file. You need to add an extra line-feed at the end of the file. There are two ways you can set about this: Open the file in a text editor and add an extra line. Then import the file to R.

Which of the following commands are used to read and write CSV files in R?

The CSV file to be read should be either present in the current working directory or the directory should be set accordingly using the setwd(…) command in R. The CSV file can also be read from a URL using read. csv() function.


The message indicates that the last line of the file doesn't end with an End Of Line (EOL) character (linefeed (\n) or carriage return+linefeed (\r\n)). The original intention of this message was to warn you that the file may be incomplete; most datafiles have an EOL character as the very last character in the file.

The remedy is simple:

  1. Open the file
  2. Navigate to the very last line of the file
  3. Place the cursor the end of that line
  4. Press return
  5. Save the file

The problem is easy to resolve; it's because the last line MUST be empty.

Say, if your content is

line 1,
line2

change it to

line 1,
line2
(empty line here)

Today I met this kind problem, when I was trying to use R to read a JSON file, by using command below:

json_data<-fromJSON(paste(readLines("json01.json"), collapse=""))

; and I resolve it by my above method.


Are you really sure that you selected the .csv file and not the .xls file? I can only reproduce the error if I try to read in an .xls file. If I try to read in a .csv file or any other text file, it's impossible to recreate the error you get.

> Data <- read.table("test.csv",header=T,sep=",")
> Data <- read.table("test.xlsx",header=T,sep=",")
Warning message:
In read.table("test.xlsx", header = T, sep = ",") :
  incomplete final line found by readTableHeader on 'test.xlsx'

readTableHead is the c-function that gives the error. It tries to read in the first n lines (standard the first 5 ) to determine the type of the data. The rest of the data is read in using scan(). So the problem is the format of the file.

One way of finding out, is to set the working directory to the directory where the file is. That way you see the extension of the file you read in. I know on Windows it's not shown standard, so you might believe it's csv while it isn't.

The next thing you should do, is open the file in Notepad or Wordpad (or another editor) and check that the format is equivalent to my file test.csv:

Test1,Test2,Test3
1,1,1
2,2,2
3,3,3
4,4,
5,5,
,6,

This file will give you the following dataframe :

> read.table(testfile,header=T,sep=",")
  Test1 Test2 Test3
1     1     1     1
2     2     2     2
3     3     3     3
4     4     4    NA
5     5     5    NA
6    NA     6    NA

The csv format saved by excel seperates all cells with a comma. Empty cells just don't have a value. read.table() can easily deal with this, and recognizes empty cells just fine.


Use readLines() (with warn = FALSE) to read the file into a character vector first.

After that use the text = option to read the vector into a data frame with read.table()

    pheasant <- read.table( 
        text = readLines(file.choose(), warn = FALSE), 
        header = TRUE,  
        sep = "," 
    )

I realized that several answers have been provided but no real fix yet.

The reason, as mentioned above, is a "End of line" missing at the end of the CSV file.

While the real Fix should come from Microsoft, the walk around is to open the CSV file with a Text-editor and add a line at the end of the file (aka press return key). I use ATOM software as a text/code editor but virtually all basic text editor would do.

In the meanwhile, please report the bug to Microsoft.

Question: It seems to me that it is a office 2016 problem. Does anyone have the issue on a PC?