How can I nest an arbitrary number of Python file context managers?

Question

I want to take an arbitrary number of paths that represent nested tar archives, and perform an operation on the innermost archive. The trouble is, the nesting can be arbitrary, so the number of context managers I need is also arbitrary.

Take, for example:

ARCHIVE_PATH = "path/to/archive.tar"

INNER_PATHS = (
    "nested/within/archive/one.tar",
    "nested/within/archive/two.tar",
    # Arbitary number of these
)

def list_inner_contents(archive_path, inner_paths):
    with TarFile(archive_path) as tf1:
        with TarFile(fileobj=tf1.extractfile(inner_paths[0])) as tf2:
            with TarFile(fileobj=tf2.extractfile(inner_paths[1])) as tf3:
                # ...arbitary level of these!
                return tfX.getnames()

contents = list_inner_contents(ARCHIVE_PATH, INNER_PATHS))

I can't use the with statement's nesting syntax because there could be any number of levels to nest. I can't use contextlib.nested because the docs say right there:

...using nested() to open two files is a programming error as the first file will not be closed promptly if an exception is thrown when opening the second file.

Is there a way to use language constructs to do this, or do I need to manually manage my own stack of open file objects?

Answer 1

For this case you may use recursion. It feels to be most natural for the case (of course if there's no special treatment in Python yet):

ARCHIVE_PATH = "path/to/archive.tar"

INNER_PATHS = [
    "nested/within/archive/one.tar",
    "nested/within/archive/two.tar",
    # Arbitary number of these
]

def list_inner_contents(archive_path, inner_paths):
    def rec(tf, rest_paths):
        if not rest_paths:
            return tf.getnames()

        with TarFile(fileobj=tf.extractfile(rest_paths[0])) as tf2:
            return rec(tf2, rest_paths[1:])

    with TarFile(archive_path) as tf:
        try:
            return rec(tf, inner_paths)
        except RuntimeError:
            # We come here in case the inner_paths list is too long
            # and we go too deeply in the recursion
            return None