Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

GZipStream decompression performance is poor

I have a .NET 2.0 WinForms app that connects to a backend WAS server. I am using GZipStream to decode data coming back from a HttpWebRequest call made to the server. The data returned is compressed CSV, which Apache is compressing. The entire server stack is Hibernate-->EJB-->Spring-->Apache.

For small responses, the performance is fine (<50ms). When I get a response >150KB, it takes more than 60 seconds to decompress. The majority of the time seems to be spent in the GZipStream constructor.

This is the code showing where I get the response stream from the HttpWebResponse call:

using (Stream stream = this.Response.GetResponseStream())
{
 if (this.CompressData && this.Response.ContentEncoding == "gzip")
 {
        // Decompress the response
  byte[] b = Decompress(stream);
  this.ResponseBody = encoding.GetString(b);
    }
 else
 {
  // Just read the stream as a string
  using (StreamReader sr = new StreamReader(stream))
  {
   this.ResponseBody = sr.ReadToEnd();
  }
 }
}

Edit 1

Based on the comment from Lucero, I modified the Decompress method to the following, but I do not see any performance benefit from loading the ResponseStream into a MemoryStream before instantiating the GZipStream.

private static byte[] Decompress(Stream stream)
{
 using (MemoryStream ms = new MemoryStream())
 {
  byte[] buffer = new byte[4096];
  int read = 0;

  while ((read = stream.Read(buffer, 0, buffer.Length)) > 0)
  {
   ms.Write(buffer, 0, read);
  }

  ms.Seek(0, SeekOrigin.Begin);

  using (GZipStream gzipStream = new GZipStream(ms, CompressionMode.Decompress, false))
  {
   read = 0;
   buffer = new byte[4096];

   using (MemoryStream output = new MemoryStream())
   {
    while ((read = gzipStream.Read(buffer, 0, buffer.Length)) > 0)
    {
     output.Write(buffer, 0, read);
    }

    return output.ToArray();
   }
  }
 }
}

Based on the code above, can anyone see any issues? This seems quite basic to me, but it's driving me nuts.

Edit 2

I profiled the application using ANTS Profiler, and during the 60s of decompression, the CPU is near zero and the memory usage does not change.

Edit 3

The actual slowdown appears to be during the read of

this.Response.GetResponseStream
The entire 60s is spent loading the response stream into the MemoryStream. Once it's there, the call to GZipStream is quick.
Edit 4

I found that using HttpWebRequest.AutomaticDecompression exhibits the same performance issue, so I'm closing this question.

like image 784
Armbrat Avatar asked Nov 05 '22 19:11

Armbrat


2 Answers

Try first loading the data into a MemoryStream and then decompress the MemoryStream...

like image 119
Lucero Avatar answered Nov 13 '22 23:11

Lucero


DotNetZip has a GZipStream class that can be used as a drop-in replacement for the System.IO.Compression.GZipStream.

DotNetZip is free.

NB: If you are only doing GZipStream, then you need the Ionic.Zlib.dll, not the Ionic.Zip.dll.

like image 40
Cheeso Avatar answered Nov 14 '22 01:11

Cheeso