how to remove '\xe2' from a list

Question

I am new to python and am using it to use nltk in my project.After word-tokenizing the raw data obtained from a webpage I got a list containing '\xe2' ,'\xe3','\x98' etc.However I do not need these and want to delete them.

I simply tried

if '\x' in a

and

if a.startswith('\xe')

and it gives me an error saying invalid \x escape

But when I try a regular expression

re.search('^\\x',a)

i get

Traceback (most recent call last):
File "<pyshell#83>", line 1, in <module>
print re.search('^\\x',a)
File "C:\Python26\lib\re.py", line 142, in search
return _compile(pattern, flags).search(string)
File "C:\Python26\lib\re.py", line 245, in _compile
raise error, v # invalid expression
error: bogus escape: '\\x'

even re.search('^\\x',a) is not identifying it.

I am confused by this,even googling didnt help(I might be missing something).Please suggest any simple way to remove such strings from the list and what was wrong with the above.

Thanks in advance!

Answer 1

You can use unicode(a, 'ascii', 'ignore') to remove all non-ascii characters in the string at once.