when writing to csv file writerow fails with UnicodeEncodeError

Question

I have the line:

c.writerow(new_values)

That writes a number of values to a csv file. Normally it is working fine but sometimes it throws an exception and doesn't write the line in the csv file. I have no idea how I can find out why.

This is my exception handling right now:

        try:
            c.writerow(new_values)
        except:
            print()
            print ("Write Error: ", new_values)

I commented out my own exception and it says:

    return codecs.charmap_encode(input,self.errors,encoding_table)[0]
UnicodeEncodeError: 'charmap' codec can't encode character '\u03b1' in position 14: character maps to <undefined>

Answer 1

Ok, I solved it by myself:

I just had to add ", encoding='utf-8'" to my csv.writer line:

c = csv.writer(open("Myfile.csv", 'w',  newline='', encoding='utf-8'))

Answer 2

the csv module in python is notorious for not handling unicode characters well. Unless all characters fall in the ascii codec you probably won't be able to write the row. There is a (somewhat) drop in replacement called unicodecsv that you may want to look into. https://pypi.python.org/pypi/unicodecsv