How to create a temporary file that can be read by a subprocess?

Question

I'm writing a Python script that needs to write some data to a temporary file, then create a subprocess running a C++ program that will read the temporary file. I'm trying to use NamedTemporaryFile for this, but according to the docs,

Whether the name can be used to open the file a second time, while the named temporary file is still open, varies across platforms (it can be so used on Unix; it cannot on Windows NT or later).

And indeed, on Windows if I flush the temporary file after writing, but don't close it until I want it to go away, the subprocess isn't able to open it for reading.

I'm working around this by creating the file with delete=False, closing it before spawning the subprocess, and then manually deleting it once I'm done:

fileTemp = tempfile.NamedTemporaryFile(delete = False) try:     fileTemp.write(someStuff)     fileTemp.close()     # ...run the subprocess and wait for it to complete... finally:     os.remove(fileTemp.name) 

This seems inelegant. Is there a better way to do this? Perhaps a way to open up the permissions on the temporary file so the subprocess can get at it?

Answer 1

Since nobody else appears to be interested in leaving this information out in the open...

tempfile does expose a function, mkdtemp(), which can trivialize this problem:

try:     temp_dir = mkdtemp()     temp_file = make_a_file_in_a_dir(temp_dir)     do_your_subprocess_stuff(temp_file)     remove_your_temp_file(temp_file) finally:     os.rmdir(temp_dir) 

I leave the implementation of the intermediate functions up to the reader, as one might wish to do things like use mkstemp() to tighten up the security of the temporary file itself, or overwrite the file in-place before removing it. I don't particularly know what security restrictions one might have that are not easily planned for by perusing the source of tempfile.

Anyway, yes, using NamedTemporaryFile on Windows might be inelegant, and my solution here might also be inelegant, but you've already decided that Windows support is more important than elegant code, so you might as well go ahead and do something readable.

Answer 2

According to Richard Oudkerk

(...) the only reason that trying to reopen a NamedTemporaryFile fails on Windows is because when we reopen we need to use O_TEMPORARY.

and he gives an example of how to do this in Python 3.3+

import os, tempfile  DATA = b"hello bob"  def temp_opener(name, flag, mode=0o777):     return os.open(name, flag | os.O_TEMPORARY, mode)  with tempfile.NamedTemporaryFile() as f:     f.write(DATA)     f.flush()     with open(f.name, "rb", opener=temp_opener) as f:         assert f.read() == DATA  assert not os.path.exists(f.name) 

Because there's no opener parameter in the built-in open() in Python 2.x, we have to combine lower level os.open() and os.fdopen() functions to achieve the same effect:

import subprocess import tempfile  DATA = b"hello bob"  with tempfile.NamedTemporaryFile() as f:     f.write(DATA)     f.flush()      subprocess_code = \     """import os        f = os.fdopen(os.open(r'{FILENAME}', os.O_RDWR | os.O_BINARY | os.O_TEMPORARY), 'rb')        assert f.read() == b'{DATA}'     """.replace('\n', ';').format(FILENAME=f.name, DATA=DATA)      subprocess.check_output(['python', '-c', subprocess_code]) == DATA