How to elegantly swap out (patch) a Django FileSystemStorage setting in your unit tests?

Question

I'm faced with the following problem. I have a Model that looks somewhat like this:

class Package(models.Model):
    name = models.CharField(max_length=64)
    file = models.FileField(upload_to="subdir",
                            storage=settings.PACKAGE_STORAGE,
                            null=True)

Essential in this example is the storage= argument to the FileField constructor. It is filled with a value from settings.py. In there is the following code:

from django.core.files.storage import FileSystemStorage
PACKAGE_STORAGE = FileSystemStorage(location="/var/data", base_url="/")

For production use, this works fine. But in my unit tests, uploads I make are now written to /var/data, which contains production data. I tried to swap out the PACKAGE_STORE in packages/tests.py like this

from django.conf import settings     # This is line 1
from tempfile import mkdtemp
settings.PACKAGE_STORAGE = FileSystemStorage(location=mkdtemp(), base_url="/")

# rest of the imports and testing code below

but the real problem is that before the test file is loaded, the packages app and its models have been loaded already, and therefore, the PACKAGE_STORAGE setting has been resolved before I'm able to change it in the test setup code.

Is there an elegant way to override this specific setting in a testing context?

Answer 1

Don't know if this counts as elegant, but you could use a different settings file for testing...

Something like:

# test_settings.py

from settings import *

PACKAGE_STORAGE = FileSystemStorage(location='/test/files', base_url="/")

Then run your test using the test settings, python manage.py test --settings=test_settings.

Answer 2

if you run the test through django this should work

if 'test' in sys.argv:
    settings.DEFAULT_FILE_STORAGE = FileSystemStorage(location=mkdtemp(), base_url="/")

of course after ;)

DEFAULT_FILE_STORAGE = FileSystemStorage(location="/var/data", base_url="/")

Answer 3

Override the underlying storage implementation for instances of FileField on your model dynamically:

def setUp(self):
     self._field = Package._meta.get_field_by_name('file')[0]
     self._default_storage = self._field.storage
     test_storage = FileSystemStorage(location=mkdtemp(),
                                      base_url="/")

     self._field.storage = test_storage

def tearDown(self):
     self._field = self._default_storage

Answer 4

I just solved this by adding to my custom test runner. To see how to add a custom test runner, see Defining a test runner in the Django documentation. My code looks something like this:

import shutil
import tempfile
from django.test.simple import DjangoTestSuiteRunner
from django.conf import settings

class CustomTestRunner(DjangoTestSuiteRunner):

    def setup_test_environment(self, **kwargs):
        super(CustomTestRunner, self).setup_test_environment(**kwargs)
        self.backup = {}
        self.backup['DEFAULT_FILE_STORAGE'] = settings.DEFAULT_FILE_STORAGE
        settings.DEFAULT_FILE_STORAGE = 'django.core.files.storage.FileSystemStorage'
        self.backup['MEDIA_ROOT'] = settings.MEDIA_ROOT
        self.temp_media_root = tempfile.mkdtemp(prefix="myapp-tests")
        settings.MEDIA_ROOT = self.temp_media_root

    def teardown_test_environment(self, **kwargs):
        super(CustomTestRunner, self).teardown_test_environment(**kwargs)
        for name, value in self.backup.iteritems():
            setattr(settings, name, value)

    def run_tests(self, test_labels, **kwargs):
        try:
            test_results = super(CustomTestRunner, self).run_tests(test_labels, **kwargs)
        finally:
            shutil.rmtree(self.temp_media_root, ignore_errors=True)

This overrides some of the custom test suite methods. setup_test_environment backs up the previous settings and stores them in a class attribute. The teardown_test_environment sets them back to what they were before. The run_tests method uses a try/finally block to make sure that the temporary directory is deleted after the tests, even if an exception happens.