Removing special characters(<200c> <200d> from a text file

Question

I have a text file with some special unseen character in it .Is there any way to remove such special character .I have tried using sed in vim editor but it doesnot work and moreover gedit and kwrite doesnot show these characters

ാര യാതരയകകിടെ യാതരകകാരന യാതരകകാരെ യാതരയില യാതരകള യാതരയകകിടയില യാതരാകകലി എതര എഫ എസ എന എയര എല എനന എനത എനറര എതതിയ എകസ എലലാ എനനാ എതരയാണ എനനീ എനറെ എചച എനനത എനന ദിവസം ദിനം ദളവതിര മിസസ മനന മോരസ മൌണട മകകോണ മനനില ടികകററ<200c> ടികകററ ടികകററില ടികകററിന ടികകററിലെ ടികകററകള ടികകററിനറെ ടികക ടെസററ ബകക<200c> ബകക ബാങക ബകകിംഗ ചെയയാം ചെയയണം ചെയയം ചെയയാനം ചെയയാതെ ചെയയാന ചെയയാമോ ചെയതത ചെയത ചെയത ചെയയാനായി ചെയയനന ചെയയേണട

<200c> is the special character i want to remove .I was able to see these character in vim editor using

 :set list

i ran following sed command in vim

:%s/\<200c\>//c

and

:%s/<200c>//c

but cannot remove special character .do i need to fire sed command in a different way??

Answer 1

I think that the only thing you're missing is how to enter special characters in your search patterns in vim. That would be with: ^Vu200c and ^Vu200d

For more information, please have a look here.

Answer 2

To remove special unicode characters <200c> and <200d> in vim:

:%s/\%u200c//g
:%s/\%u200d//g

Also you can use sed:

$ sed -i "s/$(echo -ne '\u200c')//g" file.txt
$ sed -i "s/$(echo -ne '\u200d')//g" file.txt