Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

UnicodeEncodeError when writing to a file

I am trying to write some strings to a file (the strings have been given to me by the HTML parser BeautifulSoup).

I can use "print" to display them, but when I use file.write() I get the following error:

UnicodeEncodeError: 'ascii' codec can't encode character u'\xa3' in position 6: ordinal not in range(128)

How can I parse this?

like image 505
Ivy Avatar asked Aug 04 '11 10:08

Ivy


People also ask

How do I write Unicode text to a text file in Python?

To write Unicode text to a text file with Python, we can call the file handle's write method with a Unicode encoded string. We define the string foo with a Unicode string. Then we open the test file with open with write permission. Next, we call f.

What can you use to encode the Unicode text and then write to the text file?

encode method does, and the result of encoding a unicode string is a bytestring (a str type.) You should either use normal open() and encode the unicode yourself, or (usually a better idea) use codecs. open() and not encode the data yourself.

How do I save a text file as Unicode?

Click the File > "Save As" menu. The "Save As" dialog box comes up. 3. Enter notepad_utf-16le as the new file name and select "Unicode" option in the Encoding field.


1 Answers

If I type 'python unicode' into Google, I get about 14 million results; the first is the official doc which describes the whole situation in excruciating detail; and the fourth is a more practical overview that will pretty much spoon-feed you an answer, and also make sure you understand what's going on.

You really do need to read and understand these sorts of overviews, however long they seem. There really isn't any getting around it. Text is hard. There is no such thing as "plain text", there hasn't been a reasonable facsimile for years, and there never really was, although we spent decades pretending there was. But Unicode is at least a standard.

You also should read http://www.joelonsoftware.com/articles/Unicode.html .

like image 190
Karl Knechtel Avatar answered Oct 23 '22 19:10

Karl Knechtel