sqlite database table is locked on tests

Question

I am trying to migrate an application from django 1.11.1 to django 2.0.1 Tests are set up to run with sqlite in memory database. But every test is failing, because sqlite3.OperationalError: database table is locked for every table. How can I find out why is it locked? Icreasing timeout setting does not help.

I am using LiveServerTestCase, so I suppose the tests must be running in a different thread than the in memory database, and it for some reason does not get shared.

Answer 1

I hit this, too. The LiveServerTestCase is multi-threaded since this got merged.

It becomes a problem for me when my app under test issues multiple requests. Then, so my speculation, the LiveServer spawns threads to handle those requests. Those requests then cause a write to the SQLite db. That in turn does not like multiple writing threads.

Funnily enough, runserver knows about --nothreading. But such an option seems to be missing for the test server.

The following snippet brought me a single-threaded test server:

class LiveServerSingleThread(LiveServerThread):
    """Runs a single threaded server rather than multi threaded. Reverts https://github.com/django/django/pull/7832"""

    def _create_server(self):

        """
        the keep-alive fixes introduced in Django 2.1.4 (934acf1126995f6e6ccba5947ec8f7561633c27f)
        cause problems when serving the static files in a stream.
        We disable the helper handle method that calls handle_one_request multiple times.
        """
        QuietWSGIRequestHandler.handle = QuietWSGIRequestHandler.handle_one_request

        return WSGIServer((self.host, self.port), QuietWSGIRequestHandler, allow_reuse_address=False)


class LiveServerSingleThreadedTestCase(LiveServerTestCase):
    "A thin sub-class which only sets the single-threaded server as a class"
    server_thread_class = LiveServerSingleThread

Then, derive your test class from LiveServerSingleThreadedTestCase instead of LiveServerTestCase.

Answer 2

It was caused by this django bug.

Answer 3

Using a file-based database during testing fixes the "table is locked" error. To make Django use a file-based database, specify it's filename as test database name:

DATABASES = {
    'default': {
        ...
        'TEST': {
            'NAME': os.path.join(BASE_DIR, 'db.sqlite3.test'),
       },
    }
}

I suppose that the timeout setting is ignored in case of in-memory database, see this comment for additional info.