Python tarfile progress

Question

Is there any library to show progress when adding files to a tar archive in python or alternativly would be be possible to extend the functionality of the tarfile module to do this?

In an ideal world I would like to show the overall progress of the tar creation as well as an ETA as to when it will be complete.

Any help on this would be really appreciated.

Answer 1

Unfortunately it doesn't look like there is an easy way to get byte by byte numbers.

Are you adding really large files to this tar file? If not, I would update progress on a file-by-file basis so that as files are added to the tar, the progress is updated based on the size of each file.

Supposing that all your filenames are in the variable toadd and tarfile is a TarFile object. How about,

from itertools import imap
from operator import attrgetter
# you may want to change this depending on how you want to update the
# file info for your tarobjs
tarobjs = imap(tarfile.getattrinfo, toadd)
total = sum(imap(attrgetter('size'), tarobjs))
complete = 0.0
for tarobj in tarobjs:
    sys.stdout.write("\rPercent Complete: {0:2.0d}%".format(complete))
    tarfile.add(tarobj)
    complete += tarobj.size / total * 100
sys.stdout.write("\rPercent Complete: {0:2.0d}%\n".format(complete))
sys.stdout.write("Job Done!")

Answer 2

Find or write a file-like that wraps a real file which provides progress reporting, and pass it to Tarfile.addfile() so that you can know how many bytes have been requested for inclusion in the archive. You may have to use/implement throttling in case Tarfile tries to read the whole file at once.