encoding issue in powershell search and replace

Question

I'm running a powershell script on XML files recursively to search and replace text. The code is working fine with searching and replacing the text. However in certain files there are other languages text like fréquentes which is getting changed to fréquentes after running the script. I've been using UTF8 encoding in the script. Any pointers on how to retain the encoading?

$content| Foreach-Object{$_ -replace 'test1' , 'testing'`
                            -replace 'test2' , 'testing' }| Out-File file.FullName -Encoding utf8   

Answer 1

You seem to be ignoring the XML file's encoding, which seems to be Latin 1. XML files specify their encoding at the start (or, if they don't, they will be autodetected as UTF-8, UTF-16, or UTF-32):

<?xml version='1.0' encoding='utf-8'?>

So it seems to me like you read the content with the correct encoding, but write the file in UTF-8 which doesn't match the declared one.

You could use the XML APIs to change the file, which may be preferable, or simply change your Out-File to

Out-File -Encoding Default

However, that can cause the encoding to differ between different computers, so careful with that. I pretty much only use it for files I know are in the system's legacy codepage, or for quick one-off scripts.