Decompress GZip string in Java

Question

I can find plenty of functions that let you decompress a GZip file, but how do I decompress a GZip string?

I'm trying to parse a HTTP response where the response body is compressed with GZip. However, the entire response is simply stored in a string so part of the string contains binary chars.

I'm attempting to use:

byte responseBodyBytes[] = responseBody.getBytes();
ByteArrayInputStream bais = new ByteArrayInputStream(responseBodyBytes); 
GZIPInputStream gzis = new GZIPInputStream(bais);

But that just throws an exception: java.io.IOException: Not in GZIP format

Answer 1

There's no such thing as a GZip string. GZip is binary, strings are text.

If you want to compress a string, you need to convert it into binary first - e.g. with OutputStreamWriter chained to a compressing OutputStream (e.g. a GZIPOutputStream)

Likewise to read the data, you can use an InputStreamReader chained to a decompressing InputStream (e.g. a GZIPInputStream).

One way of easily reading from a Reader is to use CharStreams.toString(Readable) from Guava, or a similar library.

Answer 2

Ideally you should use a high-level library to handle this stuff for you. That way whenever a new version of HTTP is released, the library maintainer hopefully does all the hard work for you and you just need the updated version of the library.

That aside, it is a nice exercise to try doing it yourself.

Lets assume you are reading an HTTP Response as a stream of bytes from a TCP socket. If there was no gzip encoding, then putting the whole response into a String could work. However the presence of a "Content-Encoding: gzip" header means the response body will (as you noted) be binary.

You can identify the start of the response body as the first byte following the first occurrence of the String sequence "\r\n\r\n" (or the 4 bytes 0x0d, 0x0a, 0x0d, 0x0a).

The gzip encoding has a special header, and you should test the first 3 body bytes for that:

                byte[] buf;  // from the HTTP Response stream
                // ... insert code here to populate buf from HTTP Response stream
                // ...
                int bodyLen = 1234;  // populate this value from 'Content-length' header
                int bodyStart = 123; // index of byte buffer where body starts
                if (bodyLen > 4 && buf[bodyStart] == 0x1f && buf[bodyStart + 1] == (byte) 0x8b && buf[bodyStart + 2] == 0x08) {
                    // gzip compressed body
                    ByteArrayInputStream bais = new ByteArrayInputStream(buf);
                    if (bodyStart > 0) bais.skip(bodyStart);

                    // Decompress the bytes
                    byte[] decompressedBytes = new byte[bodyLen * 4];
                    int decompressedDataLength = 0;
                    try {
                        // note: replace this try-catch with try-with-resources here where possible
                        GZIPInputStream gzis = new GZIPInputStream(bais);
                        decompressedDataLength = gzis.read(decompressedBytes);
                        gzis.close();
                    } catch (IOException e) {
                        e.printStackTrace();
                    }
                }

The "Not in GZIP format" error is produced by GZIPInputStream if the first 3 bytes do not match the magic GZIP header values, so testing for these will help resolve your particular issue.

There is also a CRC checksum within the GZIP format, however if that is missing or incorrect you should see a different error.