ZipExtFile to Django File

Question

I am wondering whether there is a way to upload a zip file to django web server and put the zip's files into django database WITHOUT accessing the actual file system in the process (e.g. extracting the files in the zip into a tmp dir and then load them)

Django provides a function to convert python File to Django File, so if there is a way to convert ZipExtFile to python File, it should be fine.

thanks for help!

Django model:

from django.db import models

class Foo:
    file = models.FileField(upload_to='somewhere')

Usage:

from zipfile import ZipFile 
from django.core.exceptions import ValidationError  
from django.core.files import File  
from io import BytesIO  

z = ZipFile('zipFile')
istream = z.open('subfile')
ostream = BytesIO(istream.read())
tmp = Foo(file=File(ostream))
try:
    tmp.full_clean()
except Validation, e:
    print e

Output:

{'file': [u'This field cannot be blank.']}

[SOLUTION] Solution using an ugly hack:

As correctly pointed out by Don Quest, file-like classes such as StringIO or BytesIO should represent the data as a virtual file. However, Django File's constructor only accepts the build-in file type and nothing else, although the file-like classes would have done the job as well. The hack is to set the variables in Django::File manually:

buf = bytesarray(OPENED_ZIP_OBJECT.read(FILE_NAME))
tmp_file = BytesIO(buf)
dummy_file = File(tmp_file)   # this line actually fails
dummy_file.name = SOME_RANDOM_NAME
dummy_file.size = len(buf)
dummy_file.file = tmp_file
# dummy file is now valid

Please keep commenting if you have a better solution (except for custom storage)

Answer 1

There's an easier way to do this:

from django.core.files.base import ContentFile

uploaded_zip = zipfile.ZipFile(uploaded_file, 'r')  # ZipFile

for filename in uploaded_zip.namelist():
    with uploaded_zip.open(filename) as f:  # ZipExtFile
        my_django_file = ContentFile(f.read())

Using this, you can convert a file that was uploaded to memory directly to a django file. For a more complete example, let's say you wanted to upload a series of image files inside of a zip to the file system:

# some_app/models.py
class Photo(models.Model):
    image = models.ImageField(upload_to='some/upload/path')

...

# Upload code    
from some_app.models import Photo

for filename in uploaded_zip.namelist():
    with uploaded_zip.open(filename) as f:  # ZipExtFile
        new_photo = Photo()
        new_photo.image.save(filename, ContentFile(f.read(), save=True)

Answer 2

Without knowing to much about Django, i can tell you to take a look at the "io" package. You could do something like:

from zipfile import ZipFile
from io import StringIO
zname,zipextfile = 'zipcontainer.zip', 'file_in_archive'
istream = ZipFile(zname).open(zipextfile)
ostream = StringIO(istream.read())

And then do whatever you would like to do with your "virtual" ostream Stream/File.