To Compress a big file in a ZIP with Java

Question

I have the need to compress a one Big file (~450 Mbyte) through the Java class ZipOutputStream. This big dimension causes a problem of "OutOfMemory" error of my JVM Heap Space. This happens because the "zos.write(...)" method stores ALL the file content to compress in an internal byte array before compressing it.

            origin = new BufferedInputStream(fi, BUFFER);
        ZipEntry entry = new ZipEntry(filePath);
        zos.putNextEntry(entry);

        int count;
        while ((count = origin.read(data, 0, BUFFER)) != -1)
        {
            zos.write(data, 0, count);
        }
        origin.close();

The natural solution will be to enlarge the heap memory space of the JVM, but I would like to know if there is a method to write this data in a streaming manner. I do not need an high compression rate so I could change the algorithm too.

does anyone have an idea about it?

Answer 1

According to your comment to Sam's response, you have obviously created a ZipOutputStream, which wraps a ByteArrayOutputStream. The ByteArrayOutputStream of course caches the compressed result in memory. If you want it written to disk, you have to wrap the ZipOutputStream around a FileOutputStream.

Answer 2

There's a library called TrueZip that I've used with good success in the past to do this kind of thing.

I cannot guarantee it does better on the buffering front. I do know that it does a lot of stuff with its own coding rather than depending on the JDK's Zip API.

So it's worth a try, in my opinion.