Where are the parameters for Indy's TIdCompressorZLib.CompressStream Method documented?

Question

The TIdComproessorZLib component is used for compression and decompression in the Delphi/C++ Builder Indy library. The CompressStream Method has the following definition:

public: virtual __fastcall CompressStream(TStream AInStream, TStream AOutStream, const TIdCompressionLevel ALevel, const int AWindowBits, const int AMemLevel, const int AStrategy);

The complete description of those parameters in the help file is:

CompressStream is a public overridden procedure. that implements the abstract the virtual method declared in the ancestor class.

AInStream is the stream containing the uncompressed contents used in the compression operation.

AOutStream is the stream used to store the compressed contents from the compression operation. AOutStream is cleared prior to outputting the compressed contents from the operation. When AOutStream is omitted, the stream in AInStream is cleared and reused for the output from the compression operation.

Use ALevel to indicate the desired compression level for the operation.

Use AWindowsBits and AMemLevel to control the memory footprint required to perform in-memory compression using the ZLib library.

Use AStrategy to control the RLE-encoding strategy used in the compression operation.

ALevel's values defined on the help page for TIdCompressionLevel, but I cannot find any indication of what values should be used for AWindowBits, AMemLevel, or AStrategy, which are just integers.

I looked in the source code, but CompressStream just delegates to IndyCompressStream, which is listed in the help file as:

IndyCompressStream(TStream InStream, TStream OutStream, const int level = Z_DEFAULT_COMPRESSION, const int WinBits = MAX_WBITS, const int MemLevel = MAX_MEM_LEVEL, const int Stratagy = Z_DEFAULT_STRATEGY);

The help for IndyCompressStream doesn't even list the minimal description of the parameters that CompressStream does.

I tracked down the file where (I think) those default constants mentioned in IndyCompressStream live, source\Indy10\Protocols\IdZLibHeaders.pas, and they are

  Z_DEFAULT_STRATEGY    = 0;
  Z_DEFAULT_COMPRESSION  = -1;
  MAX_WBITS = 15; { 32K LZ77 window }
  MAX_MEM_LEVEL = 9;

However, the value given for Z_DEFAULT_COMPRESSION is not even a legal value for that parameter according to the documentation for TIdCompressionLevel

Is there some documentation somewhere about what AWindowBits, AMemLevel, and AStrategy mean to this component, and what values are reasonable to use for them? Are the values listed above the actual recommended defaults? Also, the source files include "indy", "Indy10", and "indyimpl" directories. Which of those should we be using to find the source for the current Indy components?

Thanks!

Answer 1

You will need to look to the zlib documentation in zlib.h. In particular, the parameters to deflateInit2().

In nearly all cases, the only ones you should mess with are the compression level and the window bits. For window bits, you would normally leave the window size at 32K (15), but either add 16 for the gzip format (31), or negate (-15) to get the raw deflate format with no header or trailer. For some special kinds of data, you may get an improvement with a different compression strategy, e.g. image or other numerical arrays of data.

Answer 2

Thank you for the comments and answers, especially Remy and Mark. I had not realized that the Indy units were wrappers around zlib, and that the parameters were defined in the zlib library.

I was trying to create a gzip format stream for uploading to a server that was expecting gzip.

Here is the working code for gzip compression and decompression:

void __fastcall TForm1::Button1Click(TObject *Sender)
{
TStringStream* streamIn = new TStringStream(String("This is some data to compress"));
TMemoryStream* streamCompressed = new TMemoryStream;
TStringStream* streamOut = new TStringStream;

/* this also works to compress to gzip format, but you must #include <IdZlib.hpp>
CompressStreamEx(streamIn, streamCompressed, Idzlib::clDefault, zsGZip); */

// NOTE: according to docs, you can leave outstream null, and instream 
// will be replaced and reused, but I could not get that to work

IdCompressorZLib1->CompressStream(                
    streamIn,            // System::Classes::TStream* AInStream,
    streamCompressed,    // System::Classes::TStream* AOutStream,
    1,                   // const Idzlibcompressorbase::TIdCompressionLevel ALevel,
    15 + 16,             // const int AWindowBits, -- add 16 to get gzip format
    8,                   // const int AMemLevel, -- see note below
    0);                  // const int AStrategy);

    streamCompressed->Position = 0;
    IdCompressorZLib1->DecompressGZipStream(streamCompressed, streamOut);

    String out = streamOut->DataString;
    ShowMessage(out);
}

In particular, note that passing -1 for ALevel produces ZLib Error -2, Z_STREAM_ERROR which means invalid parameter, in spite of the defaults I had found. Also, AWindowBits normally ranges from 8 to 15, but adding 16 gives you a gzip format, and negative numbers give you a raw format, as described in the zlib documentation referenced by Mark Adler, one of the authors of the zlib library. I changed AMemLevel from Indy's default per Mark Adler's comment.

Also, as noted the CompressStreamEx function will produce gzip compression using the parameters included in the comments above.

The above was tested in RAD Studio XE3. Thanks again for your help!