How to open a list of files in Python

Question

I'm reading data file (text), and generating a number of reports, each one is written to a different output file (also text). I'm opening them the long way:

fP = open('file1','w')

invP = open('inventory','w')

orderP = open('orders','w')

... and so on, with a corresponding group of close() lines at the end.

If I could open them with a for loop, using a list of fP names and file names, I could guarantee closing the same files.

I tried using a dictionary of fp:filename, but that [obviously] didn't work, because either the fP variable is undefined, or a string 'fP' isn't a good file object name.

Since these are output files, I probably don't need to check for open errors - if I can't open one or more, I can't go on anyway.

Is there any way to open a group of files (not more than 10 or so) from a list of names, in a loop?

Answer 1

Good news! Python 3.3 brings in a standard safe way to do this:

contextlib.ExitStack

From the docs:

Each instance maintains a stack of registered callbacks that are called in reverse order when the instance is closed.
(...)
Since registered callbacks are invoked in the reverse order of registration, this ends up behaving as if multiple nested with statements had been used with the registered set of callbacks.

Here's an example how to use it:

from contextlib import ExitStack

with ExitStack() as stack:
    files = [
        stack.enter_context(open(filename))
        for filename in filenames
    ]
    # ... use files ...

When the code leaves the with statement, all files that have already been opened will be closed.

This way you also know that if 2 files get opened and then third file fails to open, the two already-opened files will be closed correctly. Also if an exception is raised anytime inside the with block, you'll see correct cleanup.

Answer 2

Yes, you can use a list comprehension:

filenames = ['file1.txt', 'file2.txt', 'file3.txt'...]
filedata = {filename: open(filename, 'w') for filename in filenames}

Now, all of the opened instances are saved in filedata, assigned to the name of the file.

To close them:

for file in filedata.values():
    file.close()