Delphi XE and ZLib Problems

Question

I'm in Delphi XE and I'm having some problems with ZLib routines...

I'm trying to compress some strings (and encode it to send it via a SOAP webservice -not really important-)

The string results from ZDecompressString differs used in ZCompressString.

example1:

uses ZLib;
// compressing string
// ZCompressString('1234567890', zcMax); 
// compressed string ='xÚ3426153·°4'

// Uncompressing the result of ZCompressString, don't return the same:
// ZDecompressString('xÚ3426153·°4'); 
// uncompressed string = '123456789'

if '1234567890' <> ZDecompressString(ZCompressString('1234567890', zcMax)) then
  ShowMessage('Compression/Decompression fails');

example2:

Uses ZLib;
// compressing string
// ZCompressString('12345678901234567890', zcMax) 
// compressed string ='xÚ3426153·°40„³'

// Uncompressing the result of ZCompressString, don't return the same:
// ZDecompressString('xÚ3426153·°40„³') 
// uncompressed string = '12345678901'

if '12345678901234567890' <> ZDecompressString(ZCompressString('12345678901234567890', zcMax)) then
  ShowMessage('Compression/Decompression fails');

the functions used are from some other posts about compressing and deCompressing

function TForm1.ZCompressString(aText: string; aCompressionLevel: TZCompressionLevel): string;
var
  strInput,
  strOutput: TStringStream;
  Zipper: TZCompressionStream;
begin
  Result:= '';
  strInput:= TStringStream.Create(aText);
  strOutput:= TStringStream.Create;
  try
    Zipper:= TZCompressionStream.Create(strOutput, aCompressionLevel);
    try
      Zipper.CopyFrom(strInput, strInput.Size);
    finally
      Zipper.Free;
    end;
    Result:= strOutput.DataString;
  finally
    strInput.Free;
    strOutput.Free;
  end;
end;

function TForm1.ZDecompressString(aText: string): string;
var
  strInput,
  strOutput: TStringStream;
  Unzipper: TZDecompressionStream;
begin
  Result:= '';
  strInput:= TStringStream.Create(aText);
  strOutput:= TStringStream.Create;
  try
    Unzipper:= TZDecompressionStream.Create(strInput);
    try
      strOutput.CopyFrom(Unzipper, Unzipper.Size);
    finally
      Unzipper.Free;
    end;
    Result:= strOutput.DataString;
  finally
    strInput.Free;
    strOutput.Free;
  end;
end;

Where I was wrong?

Someone else have same problems??

Answer 1

ZLib, like all compression codes I know, is a binary compression algorithm. It knows nothing of string encodings. You need to supply it with byte streams to compress. And when you decompress, you are given back byte streams.

But you are working with strings, and so need to convert between encoded text and byte streams. The TStringStream class is doing that work in your code. You supply the string stream instance a text encoding when you create it.

Only your code does not supply an encoding. And so the default local ANSI encoding is used. And here's the first problem. That is not a full Unicode encoding. As soon as you use characters outside your local ANSI codepage the chain breaks down.

Solve that problem by supplying an encoding when you create string stream instances. Pass the encoding to the TStringStream constructor. A sound choice is TEncoding.UTF8. Pass this when creating strInput in the compressor, and strOutput in the decompressor.

Now the next and bigger problem that you face is that your compressed data may not be a meaningful string in any encoding. You might make your existing code sort of work if you switch to using AnsiString instead of string. But it's a rather brittle solution.

Fundamentally you are making the mistake of treating binary data as text. Once you compress you have binary data. My recommendation is that you don't attempt to interpret the compressed binary as text. Leave it as binary. Compress to a TBytesStream. And decompress from a TBytesStream. So the compressor function returns TBytes and the decompressor receives that same TBytes.

If, for some reason, you must compress to a string, then you must encode the compressed binary. Do that using base64. The EncdDecd unit can do that for you.

This flow for the compressor looks like this: string -> UTF-8 bytes -> compressed bytes -> base64 string. Obviously you reverse the arrows to decompress.