python tar file how to extract file into stream

Question

I am trying to extract a zipped folder but instead of directly using .extractall(), I want to extract the file into stream so that I can handle the stream myself. Is it possible to do it using tarfile? Or is there any suggestions?

Answer 1

I was unable to extractfile while network streaming a tar file, I did something like this instead:

from backports.lzma import LZMAFile
import tarfile
some_streamed_tar = LZMAFile(requests.get('http://some.com/some.tar.xz').content)
with tarfile.open(fileobj=some_streamed_tar) as tf:
    tarfileobj.extractall(path="/tmp", members=None)

And to read them:

for fn in os.listdir("/tmp"):
    with open(os.path.join(t, fn)) as f:
        print(f.read())

python 2.7.13

Answer 2

You can obtain each file from a tar file as a python file object using the .extractfile() method. Loop over the tarfile.TarFile() instance to list all entries:

import tarfile

with tarfile.open(path) as tf:
    for entry in tf:  # list each entry one by one
        fileobj = tf.extractfile(entry)
        # fileobj is now an open file object. Use `.read()` to get the data.
        # alternatively, loop over `fileobj` to read it line by line.