Python: UnicodeEncodeError: 'ascii' codec can't encode character u'\xf1' in position 78: ordinal not in range(128)

Question

I've tried two variation in solving this problem yet will lead to another error. First by trying to encode and the other trying to strip (#) assuming that was the problem being caught in this error:

"color":rcolor,"text_color":tcolor})
File "/usr/lib/python2.7/csv.py", line 152, in writerow
    return self.writer.writerow(self._dict_to_list(rowdict))
UnicodeEncodeError: 'ascii' codec can't encode character u'\xf1' in position 78: ordinal not in range(128)

My code is now looking like this:

routes = db.routes.find()
    for route in routes:
        try:
            color = route["properties"]["color"]
            color = color.strip('#')
            print(color)

            tcolor = route["properties"]["tcolor"]
            tcolor = tcolor.strip('#')
            print(tcolor)
        except KeyError:
            color = "0000FF"
            tcolor = ""

        writer.writerow({"route_id":route["route_id"],
                   "agency_id":route["properties"]["agency_id"],...,
                   "route_color":color,"route_text_color":tcolor})

I'm not quite sure anymore why it keeps getting the unicode error...

Answer 1

Try parsing your dict to have unicode values before writing to csv.

def convert(input):
    if isinstance(input, dict):
        return {convert(key): convert(value) for key, value in input.iteritems()}
    elif isinstance(input, list):
        return [convert(element) for element in input]
    elif isinstance(input, unicode):
        return input.encode('utf-8')
    else:
        return input

if __name__ == "__main__":
    utf_route = convert(route)

So if you get the error while writing to csv please try the one below

import codecs

with codecs.open("local/file1.csv", "w", encoding='utf8') as f:
    writer = csv.writer(f, delimiter=",")
    writer.writerow(data.keys())
    writer.writerow(data.values())