How to write Chinese characters to file by python

Question

I'm walking through a directory and want to write all files names into a file. Here's the piece of code

with open("c:/Users/me/filename.txt", "a") as d:
   for dir, subdirs, files in os.walk("c:/temp"):
      for f in files:
         fname = os.path.join(dir, f)
         print fname
         d.write(fname + "\n")
d.close()

The problem I have is, there are some files that are named in Chinese characters. By using print, I can see the file name correctly in console, but in the target file, it's just a mess... I've tried to open the file like open(u"c:/Users/me/filename.txt", "a"), but it did not work. I also tried to write fname.decode("utf-16"), still does not work...

Answer 1

In Python 2, it's a good idea to use codecs.open() if you're dealing with encodings other than ASCII. That way, you don't need to manually encode everything you write. Also, os.walk() should be passed a Unicode string if you're expecting non-ASCII characters in the filenames:

import codecs
with codecs.open("c:/Users/me/filename.txt", "a", encoding="utf-8") as d:
   for dir, subdirs, files in os.walk(u"c:/temp"):
      for f in files:
         fname = os.path.join(dir, f)
         print fname
         d.write(fname + "\n")

No need to call d.close(), the with block already takes care of that.

Answer 2

Use str.encode() to encode fname before you write it to the file:

d.write(fname.encode('utf8') + '\n')