Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How can I read CSV with no quote_char in ruby?

I have a TSV file with no quote chars. Whenever a \t occurs in the data, it is always to separate columns, and never a part of a column value. Whenever a " occurs, it is always a part of a column value, and never to enclose column values.

I would like to read this CSV in Ruby but it gives me

/Users/.rvm/rubies/ruby-1.9.3-p545/lib/ruby/1.9.1/csv.rb:1925:in `block (2 levels) in shift': Illegal quoting in line 9506. (CSV::MalformedCSVError)

My code is:

CSV.foreach(input_file, { :col_sep => "\t", :headers => true}) do |row|
   puts row
end

Any way to get around this problem?

like image 892
Popcorn Avatar asked Apr 30 '14 00:04

Popcorn


People also ask

Can't process the CSV illegal quoting in line?

Illegal quoting on lineThis error is caused when there is an illegal character in the CSV file that you are trying to import. To fix this, remember that your CSV file must be UTF-8 encoded. Sometimes, this error is caused by a missing or stray quote.

How do I create a CSV file in rails?

First, set up the application and seed in some data. Now, in post. rb , declare a method which will be responsible for generating data in CSV format. Depending upon your Rails version and the dependencies added, it's possible you'll need to add a require statement.


2 Answers

Turns out I could fix it by putting quote_char => "\x00" to trick it into thinking the zero byte is the quote char.

like image 117
Popcorn Avatar answered Sep 22 '22 10:09

Popcorn


The liberal_parsing option is available for cases like this. From the documentation:

When set to a true value, CSV will attempt to parse input not conformant with RFC 4180, such as double quotes in unquoted fields.

In your example this would be:

CSV.foreach(input_file, { :col_sep => "\t", :headers => true, :liberal_parsing => true }) do |row|
  puts row
end
like image 28
Will Madden Avatar answered Sep 23 '22 10:09

Will Madden