HttpClient Request like browser

Question

When I calling site www.livescore.com by HttpClient class I always getting error "500". Probably server blocked request from HttpClients.

1)There is any other method to get html from webpage?

2)How I can set the headers to get html content?

When I set headers like in browser I always get stange encoded content.

    http_client.DefaultRequestHeaders.TryAddWithoutValidation("Accept", "text/html,application/xhtml+xml,application/xml");     http_client.DefaultRequestHeaders.TryAddWithoutValidation("Accept-Encoding", "gzip, deflate");     http_client.DefaultRequestHeaders.TryAddWithoutValidation("User-Agent", "Mozilla/5.0 (Windows NT 6.2; WOW64; rv:19.0) Gecko/20100101 Firefox/19.0");     http_client.DefaultRequestHeaders.TryAddWithoutValidation("Accept-Charset", "ISO-8859-1"); 

3) How I can slove this problem? Any suggestions?

I using Windows 8 Metro Style App in C# and HttpClientClass

Answer 1

Here you go - note you have to decompress the gzip encoded-result you get back as per mleroy:

private static readonly HttpClient _HttpClient = new HttpClient();  private static async Task<string> GetResponse(string url) {     using (var request = new HttpRequestMessage(HttpMethod.Get, new Uri(url)))     {         request.Headers.TryAddWithoutValidation("Accept", "text/html,application/xhtml+xml,application/xml");         request.Headers.TryAddWithoutValidation("Accept-Encoding", "gzip, deflate");         request.Headers.TryAddWithoutValidation("User-Agent", "Mozilla/5.0 (Windows NT 6.2; WOW64; rv:19.0) Gecko/20100101 Firefox/19.0");         request.Headers.TryAddWithoutValidation("Accept-Charset", "ISO-8859-1");          using (var response = await _HttpClient.SendAsync(request).ConfigureAwait(false))         {             response.EnsureSuccessStatusCode();             using (var responseStream = await response.Content.ReadAsStreamAsync().ConfigureAwait(false))             using (var decompressedStream = new GZipStream(responseStream, CompressionMode.Decompress))             using (var streamReader = new StreamReader(decompressedStream))             {                 return await streamReader.ReadToEndAsync().ConfigureAwait(false);             }         }     } } 

call such like:

var response = await GetResponse("http://www.livescore.com/").ConfigureAwait(false); // or var response = GetResponse("http://www.livescore.com/").Result;