Menu
I'm working on a Python script that needs to create about 50 distinct temporary files, which are all appended frequently during the course of the script and merged at the end. I'm sure that the tempfile module can do what I need, but I haven't been able to figure out how from reading the documentation.
I want to use temporary files--as opposed to variables--to conserve system memory, as these data chunks grow large as the script processes tens of thousands of other files.
The following chunk of code is the hack I'm currently using to create these files (untemporarily) in an untemporary directory:
item = (string from another file) # string must id file for future use tmpfile = 'tmpfiles/' + item if item not in totalitems: totalitems.add(item) with open(tmpfile, 'w') as itemfile: output = some stuff tmpfile.write(output) else: with open(tmpfile, 'a') as itemfile: output = different stuff tmpfile.write(output) I think what I need is tempfile.NamedTemporaryFile(). According to the documentation:
That name can be retrieved from the name member of the file object.
Unfortunately, I don't understand what that means. I just need to be able to call each file again later when I run across its corresponding "item" again in the files I'm processing. I presume this is rather straight forward and I'm just being dense. In case it matters, I have versions of this script for both Python 2.7.1 and 3.2.3. I only really need for one or the other to work; I created both just as a learning exercise.
644
Source code: Lib/tempfile.py. This module creates temporary files and directories. It works on all supported platforms. TemporaryFile , NamedTemporaryFile , TemporaryDirectory , and SpooledTemporaryFile are high-level interfaces which provide automatic cleanup and can be used as context managers.
Tempfile is a Python module used in a situation, where we need to read multiple files, change or access the data in the file, and gives output files based on the result of processed data. Each of the output files produced during the program execution was no longer needed after the program was done.
If you want to write text data into a temp file, you can use the writelines() method instead. For using this method, we need to create the tempfile using w+t mode instead of the default w+b mode. To do this, a mode param can be passed to TemporaryFile() to change the mode of the created temp file.
Temporary File Paths In contrast to tempdir() , the function tempfile() returns a file relative to a specific directory. That is, the following code will return a path for a generic file within the temporary directory.
"That name can be retrieved from the name member of the file object."
means that you can get the name of the temporary file created like so:
In [4]: import tempfile In [5]: tf = tempfile.NamedTemporaryFile() In [6]: tf.name # retrieve the name of the temp file just created Out[6]: 'c:\\blabla\\locals~1\\temp\\tmptecp3i' Note: By default the file will be deleted when it is closed. However, if the delete parameter is False, the file is not automatically deleted. See the Python docs on this for more information.
Since you can retrieve the name of each of the 50 temp files you want to create, you can save them, e.g., in a list, before you use them again later (as you say). Just be sure to set the delete value accordingly so that the files don't disappear when you close them (in case you plan to close, and then later reopen them).
I explained how to create temporary filenames in more detail here Best way to generate random file names in Python
161
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
