Django REST Framework caching bug

Question

TL;DR

I am looking for a way to clear the cache after a request, or completely disable it when running tests. Django REST Framework seems to cache the results and I need a way around this.

Long version and code

Well this turned out to behave very weird as I kept testing it. In the end, I got it to work, but I really don't like my workaround and in the name of knowledge, I have to find out why this happens and how to solve this problem properly.

So, I have an APITestCase class declared like this:

class UserTests(APITestCase):

Inside this class, I have a test function for my user-list view, as I have a custom queryset depending on the permissions. To clear things up:

• a superuser can get the whole users list (4 instances returned),
• staff members cannot see superusers (3 instances returned),
• normal users can only get 1 result, their own user (1 instance returned)

The test function version that works:

def test_user_querysets(self):

    url = reverse('user-list')

    # Creating a user
    user = User(username='user', password=self.password)
    user.set_password(self.password)
    user.save()

    # Creating a second user
    user2 = User(username='user2', password=self.password)
    user2.set_password(self.password)
    user2.save()

    # Creating a staff user
    staff_user = User(username='staff_user', password=self.password, is_staff=True)
    staff_user.set_password(self.password)
    staff_user.save()

    # Creating a superuser
    superuser = User(username='superuser', password=self.password, is_staff=True, is_superuser=True)
    superuser.set_password(self.password)
    superuser.save()



    # SUPERUSER

    self.client.logout()
    self.client.login(username=superuser.username, password=self.password)

    response = self.client.get(url)

    # HTTP_200_OK
    self.assertEqual(response.status_code, status.HTTP_200_OK)

    # All users contained in list
    self.assertEqual(response.data['extras']['total_results'], 4)



    # STAFF USER

    self.client.logout()
    self.client.login(username=staff_user.username, password=self.password)

    response = self.client.get(url)

    # HTTP_200_OK
    self.assertEqual(response.status_code, status.HTTP_200_OK)

    # Superuser cannot be contained in list
    self.assertEqual(response.data['extras']['total_results'], 3)



    # REGULAR USER

    self.client.logout()
    self.client.login(username=user2.username, password=self.password)

    response = self.client.get(url)

    # HTTP_200_OK
    self.assertEqual(response.status_code, status.HTTP_200_OK)

    # Only 1 user can be returned
    self.assertEqual(response.data['extras']['total_results'], 1)

    # User returned is current user
    self.assertEqual(response.data['users'][0]['username'], user2.username)

As you see, I am testing user permissions in this order: superuser, staff, normal user. And this works, so...

Funny thing:

If I change the order of the tests, and start with normal user, staff, superuser, the tests fail. The response from the first request gets cached, and then I get the same response when I log in as staff user, so the number of results is again 1.

The version that doesn't work:

it's exactly the same as before, only the tests are made in reverse order

def test_user_querysets(self):

    url = reverse('user-list')

    # Creating a user
    user = User(username='user', password=self.password)
    user.set_password(self.password)
    user.save()

    # Creating a second user
    user2 = User(username='user2', password=self.password)
    user2.set_password(self.password)
    user2.save()

    # Creating a staff user
    staff_user = User(username='staff_user', password=self.password, is_staff=True)
    staff_user.set_password(self.password)
    staff_user.save()

    # Creating a superuser
    superuser = User(username='superuser', password=self.password, is_staff=True, is_superuser=True)
    superuser.set_password(self.password)
    superuser.save()



    # REGULAR USER

    self.client.logout()
    self.client.login(username=user2.username, password=self.password)

    response = self.client.get(url)

    # HTTP_200_OK
    self.assertEqual(response.status_code, status.HTTP_200_OK)

    # Only 1 user can be returned
    self.assertEqual(response.data['extras']['total_results'], 1)

    # User returned is current user
    self.assertEqual(response.data['users'][0]['username'], user2.username)



    # STAFF USER

    self.client.logout()
    self.client.login(username=staff_user.username, password=self.password)

    response = self.client.get(url)

    # HTTP_200_OK
    self.assertEqual(response.status_code, status.HTTP_200_OK)

    # Superuser cannot be contained in list
    self.assertEqual(response.data['extras']['total_results'], 3)



    # SUPERUSER

    self.client.logout()
    self.client.login(username=superuser.username, password=self.password)

    response = self.client.get(url)

    # HTTP_200_OK
    self.assertEqual(response.status_code, status.HTTP_200_OK)

    # All users contained in list
    self.assertEqual(response.data['extras']['total_results'], 4)

I am working in python 2.7 with the following package versions:

Django==1.8.6
djangorestframework==3.3.1
Markdown==2.6.4
MySQL-python==1.2.5
wheel==0.24.0

UPDATE

I am using the default django cache, meaning I haven't put anything about cache in the django settings.

As suggested, I tried disabling the default Django cache:

CACHES = {
    'default': {
        'BACKEND': 'django.core.cache.backends.dummy.DummyCache',
    }
}

REST_FRAMEWORK = {
    'DEFAULT_AUTHENTICATION_CLASSES': (
        'rest_framework.authentication.SessionAuthentication',
    )
}

The problem stands on.

Even though I don't think the problem is located here, this is my UserViewSet:

api.py (the important part)

class UserViewSet(  
    mixins.RetrieveModelMixin, 
    mixins.UpdateModelMixin,
    mixins.ListModelMixin,
    viewsets.GenericViewSet
):
    queryset = User.objects.all()
    serializer_class = UserExpenseSerializer
    permission_classes = (IsAuthenticated, )
    allowed_methods = ('GET', 'PATCH', 'OPTIONS', 'HEAD')

    def get_serializer_class(self):
        if self.action == 'retrieve':
            return UserExpenseSerializer
        return UserSerializer

    def get_queryset(self):
        if(self.action == 'list'):
            return User.objects.all()
        if self.request.user.is_superuser:
            return User.objects.all()
        if self.request.user.is_staff:
            return User.objects.exclude(is_superuser=True)
        return User.objects.filter(pk = self.request.user.id)

    def list(self, request):
        filter_obj = UsersFilter(self.request)
        users = filter_obj.do_query()
        extras = filter_obj.get_extras()
        serializer = UserSerializer(users, context={'request' : request}, many=True)
        return Response({'users' : serializer.data, 'extras' : extras}, views.status.HTTP_200_OK)

filters.py

class UsersFilter:
    offset = 0
    limit = 50
    count = 0
    total_pages = 0
    filter_params = {}

    def __init__(self, request):

        if not request.user.is_superuser:
            self.filter_params['is_superuser'] = False

        if (not request.user.is_superuser and not request.user.is_staff):
            self.filter_params['pk'] = request.user.id

        # Read query params
        rpp = request.query_params.get('rpp') or 50
        page = request.query_params.get('page') or 1
        search_string = request.query_params.get('search')

        # Validate

        self.rpp = int(rpp) or 50
        self.page = int(page) or 1

        # Set filter
        set_if_not_none(self.filter_params, 'username__contains', search_string)

        # Count total results

        self.count = User.objects.filter(**self.filter_params).count()
        self.total_pages = int(self.count / self.rpp) + 1

        # Set limits
        self.offset = (self.page - 1) * self.rpp
        self.limit = self.page * self.rpp

    def get_filter_params(self):
        return self.filter_params

    def get_offset(self):
        return self.offset

    def get_limit(self):
        return self.limit

    def do_query(self):
        users = User.objects.filter(**self.filter_params)[self.offset:self.limit]
        return users

    def get_query_info(self):
        query_info = {
            'total_results' : self.count,
            'results_per_page' : self.rpp,
            'current_page' : self.page,
            'total_pages' : self.total_pages
        }
        return query_info

UPDATE 2

As Linovia pointed out, the problem was not cache or any other DRF problem, but the filter. Here's the fixed filter class:

class UsersFilter:

    def __init__(self, request):

        self.filter_params = {}
        self.offset = 0
        self.limit = 50
        self.count = 0
        self.total_pages = 0
        self.extras = {}

        if not request.user.is_superuser:
        # and so long...

Answer 1

Actually you create a new user which should make 2 users and you assert the length against 3. Not going to work even without caching.

Edit: So you actually have you issue because the use of mutables objects at the class level.

Here's the evil code:

class UsersFilter:
    filter_params = {}

    def __init__(self, request):
        if not request.user.is_superuser:
            self.filter_params['is_superuser'] = False

Which should actually be:

class UsersFilter:
    def __init__(self, request):
        filter_params = {}
        if not request.user.is_superuser:
            self.filter_params['is_superuser'] = False

Otherwise UsersFilter.filter_params will be kept from one request to another and never resets. See http://www.toptal.com/python/python-class-attributes-an-overly-thorough-guide for more details about this.