Issue with assertNumQueries

Question

I am trying to test the number of DB queries executed by an API in my Django application. So, I have a test case similar to the following:

class DatabaseQueriesTestCase(TestCase):
    scan_uri = "something"

    def test_API_1(self):
        payload = {
                    'id': 12,
                    'name': "ABCD"
                  }
        self.assertNumQueries(13, self.client.post, self.scan_uri, data=payload)

On running the test multiple times, some pass while others fail. Some fail, saying one more query was run. Why is this happening? Is this a problem with assertNumQueries ??

Note: Using Redis cache in my application..

Answer 1

I cleared my cache in setUp():

def setUp(self):
    # cleared cache
    # in my case: self.redis_conn.flushdb()

This solved the problem. You can find a helpful discussion about such a problem associated with assertNumQueries here

Many thanks to @Daniel Hepper for helping me out!!!

Answer 2

For relational database issues you need to clear the query caches first:

class DatabaseQueriesTestCase(TestCase):
    scan_uri = "something"

    def clear_query_caches(self):
        """
        Resets all caches that may prevent query execution.
        Needed to ensure deterministic behavior of ``assertNumQueries`` (or
        after external changes to some Django database records).
        """
        from django.contrib.contenttypes.models import ContentType
        from django.contrib.sites.models import Site
        ContentType.objects.clear_cache()
        Site.objects.clear_cache()

    def setUp():
        self.clear_query_caches()

    def test_API_1(self):
        payload = {
                    'id': 12,
                    'name': "ABCD"
                  }
        self.assertNumQueries(13, self.client.post, self.scan_uri, data=payload)

https://code.djangoproject.com/ticket/23746