Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

OpenCSV fails to parse double quote next to each other [duplicate]

Tags:

java

opencsv

I am using OpenCSV.

I have a CSVReader trying to parse a CSV file.
That file has quote char " and separator char , and escape char also ".

Note that the CSV contains cells like:

"ballet 24"" classes"
"\"  

which actually represent these values:

ballet 24" classes
\

Example:

"9/6/2014","3170168","123652278","Computer","2329043290","Bing and Yahoo! search","22951990789","voice lesson","Broad","0.00","0","1","3.00","0.00","0.00","0.00","7","0","",""
"9/6/2014","3170168","123652278","Smartphone","2329043291","Bing and Yahoo! search","22951990795","ballet class","Broad","0.00","0","1","1.00","0.00","0.00","0.00","0","0","",""
"9/6/2014","3170168","123652278","Smartphone","2329043291","Bing and Yahoo! search","22951990797","ballet 24"" classes","Broad","0.00","0","1","1.00","0.00","0.00","0.00","0","0","",""
"9/6/2014","3170168","123652278","Smartphone","2329043291","Bing and Yahoo! search","22951990797","ballet classes","Broad","0.00","0","1","1.00","0.00","0.00","0.00","0","0","",""
"9/6/2014","3170168","123652278","Computer","2329043291","Bing and Yahoo! search","22951990817","\","Broad","0.00","0","1","1.00","0.00","0.00","0.00","5","0","",""
"9/6/2014","3170168","123652278","Computer","2329043293","Bing and Yahoo! search","22951990850","zumba classes","Broad","0.00","0","1","7.00","0.00","0.00","0.00","5","0","",""
"9/6/2014","3170168","123652278","Smartphone","2329043293","Bing and Yahoo! search","22951990850","zumba classes","Broad","0.00","0","4","1.00","0.00","0.00","0.00","5","0","",""
"9/6/2014","3170168","123652278","Computer","2329043293","Bing and Yahoo! search","22951990874","zumba lessons","Broad","0.00","0","1","2.00","0.00","0.00","0.00","0","0","",""

My problem is that I cannot specify " for escape char to the CSVReader constructor
(i.e. make it the same as the quote char).
If I do so, the CSVReader simply goes crazy, and it reads the whole CSV line as a single CSV cell.

Has anyone else encountered this bug and how to get around it?!

like image 203
peter.petrov Avatar asked Jul 12 '16 10:07

peter.petrov


Video Answer


2 Answers

It will work if you go with the default settings for CsvReader.

Check this open bug they have: sourceforge.net/p/opencsv/bugs/83:

Actually, it works fine, just not the way you think. Its defaults are comma for separator, quote for the quote character, and backslash for the escape character. However, it understands two consecutive quote characters as an escaped quote character. So, if you just go with the defaults, it will work fine.

By default, it is able to escape double quote with double quote, but your 'true' escape character must still be something else.

So the following works:

CSVReader reader = new CSVReader(new FileReader(App.class.getClassLoader().getResource("csv.csv").getFile()), ',','"','-');
  • comma as separator
  • double quote as quote char
  • dash (any other character) as escape character

At first I put '\' as escape character, but then, your field "\" would need to be modified to escape the escape character.

like image 189
alexbt Avatar answered Jan 03 '23 16:01

alexbt


The CSVReader is not fully RFC4180 compliant. Use their newer CSV reader (RFC4180Parser):

RFC4180Parser rfc4180Parser = new RFC4180ParserBuilder().build();
CSVReaderBuilder csvReaderBuilder = new CSVReaderBuilder(
    new FileReader("input.csv"));

CSVReader reader = csvReaderBuilder
    .withCSVParser(rfc4180Parser)
    .build();

To read a String line formatted as a CSV:

String test = "ballet 24\"\" classes";
String[] columns = new RFC4180Parser().parseLine(test);

To use the reader (an alternative is reader.readNext()):

for (String[] line : reader.readAll()) {
  for (String s : line) {
    System.out.println(s);
  }
}

See http://opencsv.sourceforge.net/#rfc4180parser for more details.

Code taken from GeekPrompt

like image 42
A Jar of Clay Avatar answered Jan 03 '23 14:01

A Jar of Clay