Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Warning message: line appears to contain embedded nulls [duplicate]

Tags:

I am trying to read in a list of csv files. These csv files have ";" as its separator. After failing to read in the csv files, I have tried to cut the contents in one of the csv files into several parts and read in the values in each part to see where the problem was caused.

This method worked for me, and I have figured out a working code that works for my data:

y <- data.table(read.table(filenames[i], header = FALSE, sep = ";",                 comment.char = "", fill = TRUE, check.names = FALSE,                 blank.lines.skip = TRUE)) 

But I have encountered another problem. When I copy and paste the original data in a csv file and run the code it just works fine. However, when I try to run the same code on the original csv file, it gives me the 'embedded nulls' warning.

On the outside, the original data and the copied data look exactly the same, and they are all saved in the csv format. Therefore, it is hard for me to find what is exactly causing the warning and what is the difference between my original csv file and the copied csv file.

The data looks similar to below:

Measurement Reports export file; ; Comment;Time ;E_MW;E_PF;INV11_ACKW;INV12_ACKW;INV21_ACKW;INV22_ACKW;INV31_ACKW;INV32_ACKW;INV41_ACKW;INV42_ACKW;INV51_ACKW;INV52_ACKW;INV61_ACKW;INV62_ACKW;M1_ATEMP;M1_HUMID;M1_PYRA1S;M1_PYRA2S;M1_PYRA3S;M1_WDIREC;M1_WSPEED; ; ;00:00;-0.02  ;-0.36  ;0.00  ;0.00  ;0.00  ;0.00  ;0.00  ;0.00  ;0.00  ;0.00  ;0.00  ;0.00  ;0.00  ;0.00  ;22.32  ;82.32  ;0.00  ;0.00  ;0.00  ;234.83  ;0.00  ; ;00:01;-0.02  ;-0.36  ;0.00  ;0.00  ;0.00  ;0.00  ;0.00  ;0.00  ;0.00  ;0.00  ;0.00  ;0.00  ;0.00  ;0.00  ;22.26  ;82.57  ;0.00  ;0.00  ;0.00  ;214.93  ;0.00  ; ; ;Sum;-1.41    ;-22.10    ;0.00    ;0.00    ;0.00    ;0.00    ;0.00    ;0.00    ;0.00    ;0.00    ;0.00    ;0.00    ;0.00    ;0.00    ;1330.89    ;5098.24    ;0.00    ;0.00    ;0.00    ;11246.06    ;28.48    ; ;Mean;-0.02    ;-0.37    ;0.00    ;0.00    ;0.00    ;0.00    ;0.00    ;0.00    ;0.00    ;0.00    ;0.00    ;0.00    ;0.00    ;0.00    ;22.18    ;84.97    ;0.00    ;0.00    ;0.00    ;187.43    ;0.47    ; ; 

Any help would be appreciated. Thank you.

like image 255
user3634755 Avatar asked Jul 14 '14 10:07

user3634755


Video Answer


1 Answers

The solution, as posted in a comment by @G. Grothendieck, was to use the fileEncoding= argument to specify the correct encoding, which turned out to be either UTF16LE or UCS-2LE according to the OP.

Writing this as an answer so that is not lost to the comments.

like image 198
Benjamin Avatar answered Oct 08 '22 00:10

Benjamin