Reading from CSV: delimiter must be a string, not unicode

Question

I had a working routine (after a few helpful folks gave me some critical advice in this thread) creating model instances from a CSV file. Up to now I have been using Python 2.7 and made sure no special characters appeared anywhere. Currently I need to move to Unicode. I added

# -*- coding: utf-8 -*-

at the top of my files and everything is working nicely (I can use special characters in my code and comments), save for the CSV reader routine. Namely, the shell objects to this part:

dataReader = csv.reader(open(filename), delimiter=';', quotechar='"')

which was working before, with

TypeError: "delimiter" must be string, not unicode

After reading some older questions I switched to

dataReader = csv.reader(open(filename), delimiter=str(u';'), quotechar=str(u'"'))

to enforce the fact that the delimiter would be a string, but I'm getting exactly the same error. What am I doing wrong?

Answer 1

Your default encoding is probably not the most appropriate.

Specify the encoding like this :

dataReader = csv.reader(open(filename), delimiter=str(u';').encode('utf-8'), quotechar=str(u'"').encode('utf-8'))

Answer 2

Happened to me when I switched the code from a file without from __future__ import unicode_literals to one that had it. (python 2.7)

It changed the default encoding for string and messed with the existing code.

Fixed it by changing from:

# worked before using unicode_literals
writer = csv.writer(csvfile, delimiter=';', quotechar='"')  

to

# worked when using unicode_literals
writer = csv.writer(csvfile, delimiter=str(';'), quotechar=str('"'))