How to make the python interpreter correctly handle non-ASCII characters in string operations?

Question

I have a string that looks like so:

6 918 417 712 

The clear cut way to trim this string (as I understand Python) is simply to say the string is in a variable called s, we get:

s.replace('Â ', '') 

That should do the trick. But of course it complains that the non-ASCII character '\xc2' in file blabla.py is not encoded.

I never quite could understand how to switch between different encodings.

Here's the code, it really is just the same as above, but now it's in context. The file is saved as UTF-8 in notepad and has the following header:

#!/usr/bin/python2.4 # -*- coding: utf-8 -*- 

The code:

f = urllib.urlopen(url)  soup = BeautifulSoup(f)  s = soup.find('div', {'id':'main_count'})  #making a print 's' here goes well. it shows 6Â 918Â 417Â 712  s.replace('Â ','')  save_main_count(s) 

It gets no further than s.replace...

Answer 1

Throw out all characters that can't be interpreted as ASCII:

def remove_non_ascii(s):     return "".join(c for c in s if ord(c)<128) 

Keep in mind that this is guaranteed to work with the UTF-8 encoding (because all bytes in multi-byte characters have the highest bit set to 1).