HttpWebResponse - Encoding problem

Question

I have a problem with encoding. When I get site's source code I have:

I set encoding to UTF8 like this:

StreamReader reader = new StreamReader(response.GetResponseStream(), Encoding.UTF8);
            string sourceCode = reader.ReadToEnd();

Thanks for your help!

Answer 1

Try to use the encoding specified:

Encoding encoding;
try
{
    encoding = Encoding.GetEncoding(response.CharacterSet);
}
catch (ArgumentException)
{
    // Cannot determine encoding, use dafault
    encoding = Encoding.UTF8;
}

StreamReader reader = new StreamReader(response.GetResponseStream(), encoding);
string sourceCode = reader.ReadToEnd();

If you are accepting gzip somehow, this may help: (Haven't tried it myself and admittedly it doesn't make much sense since your encoding is not gzip?!)

request.Headers.Add(HttpRequestHeader.AcceptEncoding, "gzip,deflate");
request.AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate;

Answer 2

I had the same issue, I tried changing encoding, from the source to the result, and I got nothing. in the end, I come across a thread that leads me to the following... Take look here... .NET: Is it possible to get HttpWebRequest to automatically decompress gzip'd responses?

you need to use the following code, before retrieving the response from the request.

rqst.AutomaticDecompression = DecompressionMethods.Deflate | DecompressionMethods.GZip;

since once we use accept-encoding 'gzip' or 'deflate', the data get compressed, and turn into data unreadable by us. so we need to decompress them.