Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Check for valid utf8 string in Python

I'm reading filenames from file system and I want to send them as JSON encoded array. The problem is that files on file system can be stored in invalid encoding, and I need to handle this situation to omit invalid filenames before passing it to json.dump, otherwise it will fail.

Is there a way to check that my string (filename) contains valid utf-8 chars?

like image 555
troex Avatar asked Mar 10 '11 11:03

troex


1 Answers

How about trying the following?

valid_utf8 = True
try:
    filename.decode('utf-8')
except UnicodeDecodeError:
    valid_utf8 = False

... based on an answer to a similar question here: How to write a check in python to see if file is valid UTF-8?

like image 173
Mark Longair Avatar answered Nov 08 '22 22:11

Mark Longair