append files to an archive without reading/rewriting the whole archive

Question

I've got many files that I want to store in a single archive file. My first approach was to store the files in a gzipped tarball. The problem is, that I've to rewrite the whole archive if a single file is added. I could get rid of the gzip compression, but adding a file would still be expensive.

What other archive format would you suggest that allows fast append operations?

Answer 1

The ZIP file format was designed to allow appends without a total re-write and is ubiquitous, even on Unix.

Answer 2

ZIP and TAR fomats (and the old AR format) allow file append without a full rewrite. However:

• The Java archive classes DO NOT support this mode of operation.
• File append is likely to result in multiple copies of a file in the archive if you append an existing file.
• The ZIP and AR formats have a directory that needs to be rewritten following a file append operation. The standard utilities take precautions when rewriting the directory, but it is possible in theory that you may end up with an archive with a missing or corrupted directory if the append fails.