How to used NamedTemporaryFile (and when is it closed?)

Question

I'm trying to write a series of functions that writes to a temporary file, then does stuff with the written file. I'm trying to understand how the file is dealt with.

What I'd like to do in the abstract is:

def create_function(inputs):
    # create temp file, write some contents

def function1(file):
    # do some stuff with temp file

def function2(file):
    # do some other stuff with temp file

So that I can do something like:

my_file = create_function(my_inputs)
function1(my_file)
function2(my_file)

So here's what I've actually done:

def db_cds_to_fna(collection, open_file):
    """
    This pulls data from a mongoDB and writes it to a temporary file - just writing an arbitrary string doesn't alter my question (I'm pretty sure)
    """
    for record in db[collection].find({"type": "CDS"}):
        open_file.write(">{}|{}|{}\n{}\n".format(
            collection,
            record["_id"],
            record["annotation"],
            record["dna_seq"]
            )
        )

    return open_file.name

def check_file(open_file):
    lines = 0
    for line in open_file:
        if lines < 5:
            print line
            lines += 1
        else:
            break

With this code, if I run the following:

from tempfile import NamedTemporaryFile
tmp_file = NamedTemporaryFile()
tmp_fna =  db_cds_to_fna('test_collection', tmp_file)

check_file(tmp_file)

This code runs, but doesn't actually print anything. But the file is clearly there and written, because if I run print Popen(['head', tmp_fna], stdout=PIPE)[0], I get the expected beginning of the file. Or, if I change check_file() to accept the tmp_file.name and do with open(tmp_file.name, 'r')... inside the function, it works.

So question 1 is - why can I write to the tmp_file, but can't read it from a different function without re-opening it?

Now, what I'd really like to do is have the tmp_file = NamedTemporaryFile() inside the db_cds_to_fna() function, but when I try this and run:

tmp_fna =  db_cds_to_fna('test_collection')
check_file(tmp_file)

I get an error No such file or folder

So question 2 is: is there any way to keep the temporary file around for another function to use? I know how to just write a file to a specified path and then delete it, but I suspect there's a built in way to do this and I'd like to learn.

Answer 1

You're writing to the file, but then you're attempting to read it from the end of your writes. Add a seek before you start reading, to go back to the beginning of the file:

def check_file(open_file):
    lines = 0
    open_file.seek(0)
    for line in open_file:
        if lines < 5:
            print line
            lines += 1
        else:
            break

For your second question, note that NamedTemporaryFile works like TemporaryFile in that:

It will be destroyed as soon as it is closed (including an implicit close when the object is garbage collected).

If you open the file in a function and then return, the file goes out of scope and will be closed and garbage collected. You'll need to keep a reference to the file object alive in order to keep it from being collected. You can do this by returning the file object from the function (and making sure you assign it to something). Here's a trivial example:

def mycreate():
    return NamedTemporaryFile()
def mywrite(f, i):
    f.write(i)
def myread(f):
    f.seek(0)
    return f.read()

f = mycreate()        # 'f' is now a reference to the file created in the function, 
                      # which will keep it from being garbage collected
mywrite(f, b'Hi')
myread(f)