How to clean up temporary file used with send_file?

Question

I'm currently developing a server side json interface where several temporary files are manipulating during requests.

My current solution for cleaning up these files at the end of the request looks like this:

@app.route("/method",methods=['POST'])
def api_entry():
    with ObjectThatCreatesTemporaryFiles() as object:
        object.createTemporaryFiles()
        return "blabalbal"

In this case, the cleanup takes lace in object.__exit__()

However in a few cases I need to return a temporary files to the client, in which case the code looks like this:

@app.route("/method",methods=['POST'])
def api_entry():
    with ObjectThatCreatesTemporaryFiles() as object:
        object.createTemporaryFiles()
        return send_file(object.somePath)

This currently does not work, because when I the cleanup takes place flask is in the process of reading the file and sending it to the client. ¨ How can I solve this?

Edit: I Forgot to mention that the files are located in temporary directories.

Answer 1

The method I've used is to use weak-references to delete the file once the response has been completed.

import shutil
import tempfile
import weakref

class FileRemover(object):
    def __init__(self):
        self.weak_references = dict()  # weak_ref -> filepath to remove

    def cleanup_once_done(self, response, filepath):
        wr = weakref.ref(response, self._do_cleanup)
        self.weak_references[wr] = filepath

    def _do_cleanup(self, wr):
        filepath = self.weak_references[wr]
        print('Deleting %s' % filepath)
        shutil.rmtree(filepath, ignore_errors=True)

file_remover = FileRemover()

And in the flask call I had:

@app.route('/method')
def get_some_data_as_a_file():
    tempdir = tempfile.mkdtemp()
    filepath = make_the_data(dir_to_put_file_in=tempdir)
    resp = send_file(filepath)
    file_remover.cleanup_once_done(resp, tempdir)
    return resp

This is quite general and as an approach has worked across three different python web frameworks that I've used.

Answer 2

If you are using Flask 0.9 or greater you can use the after_this_request decorator:

@app.route("/method",methods=['POST'])
def api_entry():
    tempcreator = ObjectThatCreatesTemporaryFiles():
    tempcreator.createTemporaryFiles()

    @after_this_request
    def cleanup(response):
        tempcreator.__exit__()
        return response

    return send_file(tempcreator.somePath)

EDIT

Since that doesn't work, you could try using cStringIO instead (this assumes that your files are small enough to fit in memory):

@app.route("/method", methods=["POST"])
def api_entry():
    file_data = dataObject.createFileData()
    # Simplest `createFileData` method:  
    # return cStringIO.StringIO("some\ndata")
    return send_file(file_data,
                        as_attachment=True,
                        mimetype="text/plain",
                        attachment_filename="somefile.txt")

Alternately, you could create the temporary files as you do now, but not depend on your application to delete them. Instead, set up a cron job (or a Scheduled Task if you are running on Windows) to run every hour or so and delete files in your temporary directory that were created more than half an hour before.