How to concat two or more gzip files/streams

Question

I want to concat two or more gzip streams without recompressing them.

I mean I have A compressed to A.gz and B to B.gz, I want to compress them to single gzip (A+B).gz without compressing once again, using C or C++.

Several notes:

• Even you can just concat two files and gunzip would know how to deal with them, most of programs would not be able to deal with two chunks.
• I had seen once an example of code that does this just by decompression of the files and then manipulating original and this significantly faster then normal re-compression, but still requires O(n) CPU operation.
• Unfortunaly I can't found this example I had found once (concatenation using decompression only), if someone can point it I would be greatful.

Note: it is not duplicate of this because proposed solution is not fits my needs.

Clearification edit:

I want to concate several compressed HTML pices and send them to browser as one page, as per request: "Accept-Encoding: gzip", with respnse "Content-Encoding: gzip"

If the stream is concated as simple as cat a.gz b.gz >ab.gz, Gecko (firefox) and KHTML web engines gets only first part (a); IE6 does not display anything and Google Chrome displays first part (a) correctly and the second part (b) as garbage (does not decompress at all).

Only Opera handles this well.

So I need to create a single gzip stream of several chunks and send them without re-compressing.

Update: I had found gzjoin.c in the examples of zlib, it does it using only decompression. The problem is that decompression is still slower them simple memcpy.

It is still faster 4 times then fastest gzip compression. But it is not enough.

What I need is to find the data I need to save together with gzip file in order to not run decompression procedure, and how do I find this data during compression.

Answer 1

Look at the RFC1951 and RFC1952

The format is simply a suites of members, each composed of three parts, an header, data and a trailer. The data part is itself a set of chunks with each chunks having an header and data part.

To simulate the effect of gzipping the result of the concatenation of two (or more files), you simply have to adjust the headers (there is a last chunk flag for instance) and trailer correctly and copying the data parts.

There is a problem, the trailer has a CRC32 of the uncompressed data and I'm not sure if this one is easy to compute when you know the CRC of the parts.

Edit: the comments in the gzjoin.c file you found imply that, while it is possible to compute the CRC32 without decompressing the data, there are other things which need the decompression.