Opening a data file from the media directory in Django

Question

I have an application that allows for users to upload CSV files with data which is then graphed and displayed to the user. These files are saved as media within my media folder. In my graph view however I need to open the file and process it. My problem is that I can only open files that are within my project's current working directory and any time that I attempt to upload a file from somewhere outside of that directory I get this error:

File b'TEST.CSV' does not exist

I have attempted this, but without success:

file_upload_dir = os.path.join(settings.MEDIA_ROOT, 'Data_Files')
data_file = open(os.path.join(file_upload_dir, new_file), 'rb')

The variable new_file is only the name of the file saved from in a session and not a path to that file. Data_Files is a directory within the media directory that contains the uploaded files.

My media settings for Django are

SETTINGS_DIR = os.path.dirname(__file__)
PROJECT_PATH = os.path.join(SETTINGS_DIR, os.pardir)
PROJECT_PATH = os.path.abspath(PROJECT_PATH)

MEDIA_URL = '/media/'
MEDIA_ROOT = os.path.join(PROJECT_PATH, 'media')

Is there a way to reference the media files properly from a view?

Here is the output of file_upload_dir and the location of the new file.

>>> print(file_upload_dir)
C:\\Users\\vut46744\\Desktop\\graphite_project\\media\\Data_Files

>>> print(os.path.join(file_upload_dir, new_file))
C:\\Users\\vut46744\\Desktop\\graphite_project\\media\\Data_Files\\TEST.CSV

Answer 1

Normally, you should not access files using open() in a Django app. You should use the storage API. This allows your code to play well with Django settings, and potential third party apps that augment this API. https://docs.djangoproject.com/en/1.7/topics/files/#file-storage

So here you should be doing something like

from django.core.files.storage import default_storage

f = default_storage.open(os.path.join('Data_Files', new_file), 'r')
data = f.read()
f.close()
print(data)

By the way, if you want it to be modular, it would be a good idea to have a custom storage class, allowing easy configuration and use of your app, should your requirements change. Also, that allows putting files outside of MEDIA_ROOT. This sample storage will put them in settings.UPLOADS_ROOT (and default to MEDIA_ROOT if the setting is not found).

# Put this in a storage.py files in your app
from django.conf import settings
from django.core.files.storage import FileSystemStorage, get_storage_class
from django.utils.functional import LazyObject

class UploadsStorage(FileSystemStorage):
    def __init__(self, location=None, base_url=None, *args, **kwargs):
        if location is None:
            location = getattr(settings, 'UPLOADS_ROOT', None)
        super(UploadsStorage, self).__init__(location, base_url, *args, **kwargs)
        self.base_url = None  # forbid any URL generation for uploads

class ConfiguredStorage(LazyObject):
    def _setup(self):
        storage = getattr(settings, 'UPLOADS_STORAGE', None)
        klass = UploadsStorage if storage is None else get_storage_class(storage)
        self._wrapped = klass()
uploads_storage = ConfiguredStorage()

We create a very simple storage here. It's just the regular one, but that reads its files from another directory. Then we set up a lazy object that will allow overriding that storage from settings.

So now your code becomes:

from myapp.storage import uploads_storage
f = uploads_storage.open(new_files, 'r')

And in your settings, you set UPLOADS_ROOT to whatever you like. Probably something outside your media directory. And if someday you decide to store uploads in a database instead, you can set UPLOADS_STORAGE to a database-backed storage, your code will happily use it.