Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

CSV new-line character seen in unquoted field error

Tags:

python

csv

django

People also ask

How do you handle a new line in csv?

To embed a newline in an Excel cell, press Alt+Enter. Then save the file as a . csv. You'll see that the double-quotes start on one line and each new line in the file is considered an embedded newline in the cell.

What is the newline character in csv?

New Line Characters Windows standard CR+LF or Unix-like systems (Linux, Mac) standard LF may be used as new line character.

What is the importance of new line in CSV file?

Including the newline parameter allows the csv module to handle the line endings itself - replicating the format as defined in your csv.

Do you need to open the file in Universal newline mode?

While universal newlines are automatically enabled for import they are not for opening, where you have to specifically say open(..., "U") . This is open to debate, but here are a few reasons for this design: Compatibility. Programs which already do their own interpretation of \r\n in text files would break.


It'll be good to see the csv file itself, but this might work for you, give it a try, replace:

file_read = csv.reader(self.file)

with:

file_read = csv.reader(self.file, dialect=csv.excel_tab)

Or, open a file with universal newline mode and pass it to csv.reader, like:

reader = csv.reader(open(self.file, 'rU'), dialect=csv.excel_tab)

Or, use splitlines(), like this:

def read_file(self):
    with open(self.file, 'r') as f:
        data = [row for row in csv.reader(f.read().splitlines())]
    return data

I realize this is an old post, but I ran into the same problem and don't see the correct answer so I will give it a try

Python Error:

_csv.Error: new-line character seen in unquoted field

Caused by trying to read Macintosh (pre OS X formatted) CSV files. These are text files that use CR for end of line. If using MS Office make sure you select either plain CSV format or CSV (MS-DOS). Do not use CSV (Macintosh) as save-as type.

My preferred EOL version would be LF (Unix/Linux/Apple), but I don't think MS Office provides the option to save in this format.


For Mac OS X, save your CSV file in "Windows Comma Separated (.csv)" format.


If this happens to you on mac (as it did to me):

  1. Save the file as CSV (MS-DOS Comma-Separated)
  2. Run the following script

    with open(csv_filename, 'rU') as csvfile:
        csvreader = csv.reader(csvfile)
        for row in csvreader:
            print ', '.join(row)
    

Try to run dos2unix on your windows imported files first


This is an error that I faced. I had saved .csv file in MAC OSX.

While saving, save it as "Windows Comma Separated Values (.csv)" which resolved the issue.


This worked for me on OSX.

# allow variable to opened as files
from io import StringIO

# library to map other strange (accented) characters back into UTF-8
from unidecode import unidecode

# cleanse input file with Windows formating to plain UTF-8 string
with open(filename, 'rb') as fID:
    uncleansedBytes = fID.read()
    # decode the file using the correct encoding scheme
    # (probably this old windows one) 
    uncleansedText = uncleansedBytes.decode('Windows-1252')

    # replace carriage-returns with new-lines
    cleansedText = uncleansedText.replace('\r', '\n')

    # map any other non UTF-8 characters into UTF-8
    asciiText = unidecode(cleansedText)

# read each line of the csv file and store as an array of dicts, 
# use first line as field names for each dict. 
reader = csv.DictReader(StringIO(cleansedText))
for line_entry in reader:
    # do something with your read data