Using python's csv
module, I'm trying to read some CSV data.
I'm using the code:
dialect = csv.Sniffer().sniff(csv_file.read(1024))
csv_file.seek(0)
reader = csv.reader(csv_file, dialect)
for line in reader:
...
Everything works fine except for lines containing escaped quotes:
11837,2,NULL,"\"The Take Over, The Breaks Over\"","Fall Out Boy"
Such a line is tokenized as:
['11837', '2', 'NULL', '\\The Take Over', ' The Breaks Over\\""', 'Fall Out Boy']
The dialect contains the following properties:
dialect.quotechar = "
dialect.quoting = 0
dialect.escapechar = None
dialect.delimiter = ,
dialect.doublequote = False
dialect.lineterminator = \n
Is there anything I can do besides writing my own CSV parser?
quotechar specifies the character used to surround fields that contain the delimiter character. The default is a double quote ( ' " ' ). escapechar specifies the character used to escape the delimiter character, in case quotes aren't used.
QUOTE_NONE ), the csv module uses the quotechar (which defaults to " ) to quote field. The following listing changes the quote character from double quote ( " ) to a single quote ( ' ). In this case, the csv module uses the single quote ( ' ) instead of ( " ) to quote fields containing quotechar or delimiter.
A one-character string used by the writer to escape the delimiter if quoting is set to QUOTE_NONE and the quotechar if doublequote is False . On reading, the escapechar removes any special meaning from the following character.
The csv module defines the following constants: csv.QUOTE_ALL. Instructs writer objects to quote all fields. csv.QUOTE_MINIMAL. Instructs writer objects to only quote those fields which contain special characters such as delimiter, quotechar or any of the characters in lineterminator.
If I'm not mistaken, dialect.escapechar = None
should be dialect.escapechar = '\\'
If you look at the examples in the docs, it certainly seems to suggest making that alteration
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With