How do I parse a tab delimited file that may have values across multiple lines?

Question

I have a file that is tab delimted with different data points:

"ID"    "Value"
"1" "This is a value"

I can easily extract the data from this by simply using the built-in str function split. However there are times that I run into this:

"ID"    "Value"
"1" "This is a value"
"2" "This is another
value"
"3" "Just one more"

Where the second value runs across multiple lines. How can I capture each data point in it's fullness?

Ultimately what I want is a list of dictionaries like so:

[{'ID':'1', 'Value':'This is a value'}, {'ID':'2', 'Value':'This is another\nvalue'}, {'ID':'3', 'Value':'Just one more'}]

Answer 1

import csv
r=csv.reader(open("a.tsv"), delimiter="\t", quotechar='"')
print r.next()

Here is a runnable example http://codebunk.com/b/4095452/

Answer 2

When iterating over the lines, you have two possibilities: In the default case, you are reading a new record, so you should just handle it as you would without the multi-line case. The other case is when the previous line didn’t end the record, i.e. when it didn’t end with a quote. In that case, you are still adding to the previous record. So you just need to keep track of the status of the previous record an the record itself to parse your file.

Something like this:

isNew = True
records = []
for line in file:
    if isNew:
        records.append(line.strip().split('\t'))
    else:
        records[-1][-1] += '\n' + line
    isNew = records[-1][-1].endswith('"')