How to compress a large file in Python?

Question

The problem I'm experiencing is the name of the stored file. The stored file isn't named with the original/uncompressed file name. Instead the stored file is named with the archive name (with the appended ".gz" extension).

Expected Result:

file.txt.gz           {archive name}
....file.txt          {stored file name}

Actual Result:

file.txt.gz           {archive name}
....file.txt.gz       {stored file name}

Reading through the gzip documentation (https://docs.python.org/2.7/library/gzip.html) example code:

import gzip
import shutil
with open('file.txt', 'rb') as f_in, gzip.open('file.txt.gz', 'wb') as f_out:
    shutil.copyfileobj(f_in, f_out)

How do I get the archive to store the file with the name "file.txt" instead of "file.txt.gz"?

Answer 1

You have to use gzip.GzipFile(); the shorthand gzip.open() won't do what you want.

Quoth the doc:

When fileobj is not None, the filename argument is only used to be included in the gzip file header, which may include the original filename of the uncompressed file. It defaults to the filename of fileobj, if discernible; otherwise, it defaults to the empty string, and in this case the original filename is not included in the header.

Try this:

import gzip
import shutil
with open('file.txt', 'rb') as f_in:
    with open('file.txt.gz', 'wb') as f_out:
        with gzip.GzipFile('file.txt', 'wb', fileobj=f_out) as f_out:
            shutil.copyfileobj(f_in, f_out)