HttpWebRequest & Native GZip Compression

Question

When requesting a page with Gzip compression I am getting a lot of the following errors:

System.IO.InvalidDataException: The CRC in GZip footer does not match the CRC calculated from the decompressed data

I am using native GZipStream to decompress and am looking at addressing this. With that in mind is there a work around for addressing this or another GZip library (free?) which will handle this issue properly?

I am verifying the webResponse ContentEncoding is GZIP

Update 5/11 A simplified snippit

//Caller public void SOSampleGet(string url)  {     // Initialize the WebRequest.     webRequest = (HttpWebRequest)WebRequest.Create(url);     webRequest.Method = WebRequestMethods.Http.Get;     webRequest.KeepAlive = true;     webRequest.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8";     webRequest.Headers.Add("Accept-Encoding", "gzip,deflate");     webRequest.Referer = WebUtil.GetDomain(url);      HttpWebResponse webResponse = (HttpWebResponse)webRequest.GetResponse();          using (Stream stream = GetStreamForResponse(webResponse, READTIMEOUT_CONST))     {         //use stream     } }  //Method private static Stream GetStreamForResponse(HttpWebResponse webResponse, int readTimeOut) {     Stream stream;     switch (webResponse.ContentEncoding.ToUpperInvariant())     {         case "GZIP":             stream = new GZipStream(webResponse.GetResponseStream(), CompressionMode.Decompress);             break;         case "DEFLATE":             stream = new DeflateStream(webResponse.GetResponseStream(), CompressionMode.Decompress);             break;          default:             stream = webResponse.GetResponseStream();             stream.ReadTimeout = readTimeOut;             break;         }         return stream; } 

Answer 1

What about the webrequest AutomaticDecompression Property available since .net 2? Simply add:

webRequest.AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate; 

It also adds the gzip,deflate to the accept encoding header.

See http://msdn.microsoft.com/en-us/library/system.net.httpwebrequest.automaticdecompression.aspx

Answer 2

For .NET Core things are a little more involved. A GZipStream is needed as there isn't a property (as of writing) for AutomaticCompression. See my answer here: https://stackoverflow.com/a/44508724/2421277

Code from answer:

var req = WebRequest.CreateHttp(uri);  /*  * Headers  */ req.Headers[HttpRequestHeader.AcceptEncoding] = "gzip, deflate";  /*  * Execute  */ try {     using (var resp = await req.GetResponseAsync())     {         using (var str = resp.GetResponseStream())         using (var gsr = new GZipStream(str, CompressionMode.Decompress))         using (var sr = new StreamReader(gsr))          {             string s = await sr.ReadToEndAsync();           }     } } catch (WebException ex) {     using (HttpWebResponse response = (HttpWebResponse)ex.Response)     {         using (StreamReader sr = new StreamReader(response.GetResponseStream()))         {             string respStr = sr.ReadToEnd();             int statusCode = (int)response.StatusCode;              string errorMsh = $"Request ({url}) failed ({statusCode}) on, with error: {respStr}";         }     } }