Mocking Django Storages Model ImageField backend S3

Question

I have a model with an ImageField that is backed by django-storages' S3Boto. I have a test the exercises the "upload image" view, but the fact that it is uploading the image to S3 is slowing down my test suite.

In the interest of speeding up my tests, what is the best practice for dealing with this issue? Should I mock out S3Boto? Perhaps there is a memory backed storage backend that works well for testing (automatic cleanup would be nice!)?

Answer 1

I just had this problem too. I got much faster tests by using dj-inmemorystorage.

The quick way of setting this up is by creating a test_settings.py in the same folder as your settings:

from settings import *
DEFAULT_FILE_STORAGE = 'inmemorystorage.InMemoryStorage'

...and calling ./manage.py test --settings=project.test_settings to run the tests.

My preferred way is to set up a custom test runner:

In project/test_runner.py:

from django.conf import settings
from django.test.runner import DiscoverRunner

class FastTestRunner(DiscoverRunner):
    def setup_test_environment(self):
        super(FastTestRunner, self).setup_test_environment()
        # Don't write files
        settings.DEFAULT_FILE_STORAGE = 'inmemorystorage.InMemoryStorage'
        # Bonus: Use a faster password hasher. This REALLY helps.
        settings.PASSWORD_HASHERS = (
            'django.contrib.auth.hashers.MD5PasswordHasher',
        )

Note: This also sets the PASSWORD_HASHER, because it significantly improves User creation time. This should NOT be set in production.

In project/settings.py:

TEST_RUNNER = 'project.test_runner.FastTestRunner'

The requirements:

pip install dj-inmemorystorage

UPDATE: changed from django-inmemorystorage to dj-inmemorystorage.

UPDATE 2: Removed django-discover-runner, as it's now the default test runner in django, and fixed the link to the PASSWORD_HASHER related blog post.

Answer 2

I also use S3Boto but for testing, I prefer having custom settings which include using the file system storage. You can have your custom settings declared in a file which you can then import and use in your test cases. Even so, you can mock the file storage so that the files are not actually written to disk.

Here's a sample test_settings.py

# myproject/myproject/test_settings.py

from django.test import override_settings

common_settings = override_settings(
    DEFAULT_FILE_STORAGE='django.core.files.storage.FileSystemStorage',
    PASSWORD_HASHERS=(
        'django.contrib.auth.hashers.UnsaltedMD5PasswordHasher',
    ),
)

Usage:

from django.test import TestCase

from myproject.test_settings import common_settings

@common_settings
class MyTestCase(TestCase):
    """Tests go here"""

On mocking the file system storage you can check out my answer here on SO.