Python and Django - How to use in memory and temporary files

Question

I need some examples with file operations using in memory and temporary files.

I searched a lot for a good example /tutorial and I found just basic read/write/append operations.

I need to understand how can I read in Django a file that is uploaded(an image) before the save(post) is finished.

Because Django is Python, I think is better to understand first in python.

I checked the Django documentation/examples about, but it is not very clear so I need to understand first in Python then in Django how the operations are working, not just copy and paste.

I know how to use ImageFields, upload default operation, I'm interested only in using "in memory and temporary files."

I want to use this in combination with a crop function. So the user can upload 1, 2, 3... images, and with a javascript crop script I get the coordinates in a hidden field. After javascript simulated crop I show the user a thumbnail with the crop, how will look like ratio

The user can change his mind and can edit/update or delete a file before saving.

Now depending on the file size it can be keep in memory or write.

Answer 1

When a file is uploaded Django will do one of two things: store it in memory if the file is small (< 2 MB last time I checked), or store it as a temporary file on disk if it's large. This behavior is configurable via the FILE_UPLOAD_HANDLERS setting. So, your web server and Django take care of the actual upload and storage, but it's your job to process the file before the request is over, otherwise the file is deleted.

Uploaded files are accessible through the request.FILES property. Each key in FILES will match the name of the file input on your <form>. The value is an UploadedFile object, which is a stream you can use to read the file data.

For example, say you have an <input name="img" type="file" /> and you want to detect if the image is completely white. You don't need to store the file for this, you just need to load it into memory, process it to get the result and then let it be discarded.

from PIL import Image

def some_view(request):
    if request.method == 'POST':
        img_file = request.FILES['img']

        if img_file.size > 2000000:
            return HttpResponseBadRequest()

        img = Image.open(img_file)

        # analyze the image...

Another possibility is that someone is uploading a backup file that is quite large (lets say 2 GB), and you need to store it somewhere. It's effectively the same thing, except we read the file into memory in chunks, then write each chunk to disk somewhere else so that it's saved after the request finishes.

def some_view(request):
    if request.method == 'POST':
        backup_file = request.FILES['backup_file']
        with open('some/file/name.bak', 'wb+') as destination:
            for chunk in backup_file.chunks():
                destination.write(chunk)

        # file is saved

When the request is over, the uploaded file is stored at some/file/name.bak.

Whether it's in memory or a temporary file is usually not important because the interface is the same. You can read a temporary file just like you can read an in memory file.