Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Converting problem ANSI to UTF8 C#

I have a problem with converting a text file from ANSI to UTF8 in c#. I try to display the results in a browser.

So I have a this text file with many accent character in it. Its encoded in ANSI, so I have to convert it to utf8 because in the browser instead of the accentchars appearing "?". No matter how I tried to convert to UTF8 it was still a "?". But if I convert the text file in notepad++ to utf8 then the accent chars are desplayed good.

here is a peace of encoding code that I made:

    public string Encode(string text)
    {
        // encode the string as an ASCII byte array
        byte[] myASCIIBytes = ASCIIEncoding.ASCII.GetBytes(text);

        // convert the ASCII byte array to a UTF-8 byte array
        byte[] myUTF8Bytes = ASCIIEncoding.Convert(ASCIIEncoding.ASCII, UTF8Encoding.UTF8, myASCIIBytes);

        // reconstitute a string from the UTF-8 byte array 
        return UTF8Encoding.UTF8.GetString(myUTF8Bytes);
    }

Do you have any idea why is this happening?

like image 771
JahManCan Avatar asked Sep 23 '10 12:09

JahManCan


Video Answer


2 Answers

Do you have any idea why is this happening?

Yes, you're too late. You need to specify ANSI when you read the string from file. In memory it's always Unicode (UTF16).

like image 162
Henk Holterman Avatar answered Oct 25 '22 09:10

Henk Holterman


When you convert to ASCII you immediately lose all non-English characters (including ones with accent) because ASCII has only 127 (7 bits) of characters.

You do strange manipulation. string in .net is in UTF-16, so once you return string, not byte[] this doesn't matter.

I think you should do: (I guess by ANSI you mean Latin1)

public byte[] Encode(string text)
{
    return Encoding.GetEncoding(1252).GetBytes(text);
}

Since the question was not very clear there is a reasonable remark that you might actually need this one:

public string Decode(byte[] data)
{
    return Encoding.GetEncoding(1252).GetString(data);
}
like image 20
Andrey Avatar answered Oct 25 '22 07:10

Andrey