Randomly failing tests jest and supertest Node.js

Question

so long story short I'm developing RESTapi that takes a movie title on POST request to the /movies route, fetches info about that movie from external api and saves that object to the database. On POST /comments you add a comment to the different collection but every comment has a 'movie_id' property that has an associated movie.

That's my first bigger project so I'm trying to write integrational tests. Everything is great, at least in my opinion, except 3 weird test cases that are failing just out of nowhere. Tests could pass 10 times in a row and then suddenly that weird 'jest' timer shows up and 3 cases fail.

I'm using native mongodb driver, express and jest with supertest for testing, dropping test-database BeforeAll and AfterEach, I have no idea whats the reason of that.

Timer thingy:

And after timer this shows up, failed tests:

Full source code is here GITHUB

Other failed cases:

Any ideas, tips?

Answer 1

I was in the same hell "jest parallel tests" problem and i find a solution, maybe not the best but now jest run tests in "queue mode" so when i delete datas in beforeAll my next group of tests is ready to go with "fresh" new inserted datas.

--runInBand Alias: -i. Run all tests serially in the current process, rather than creating a > worker pool of child processes that run tests. This can be useful for debugging.

jest source

So in my config.json i have :

    "scripts": {
        "test": "set NODE_ENV=test&& jest ./tests --runInBand --detectOpenHandles --forceExit",
        "server": "set NODE_ENV=development&& nodemon app.js"
    }

Answer 2

The code seems to return one entry, while the test expects zero. This looks very much like an issue with test independence: Your tests seem to depend on each other (through the database).

I would guess that one test creates the movie and then clears it again. When everything works fine, the second test does not find the movie. But, in some unforunate cases (bad timing, different execution order, ...), the database is not cleared fast enough and the second test finds the movie.

So, you should work hard on making your tests independent. This is not easy with integrated tests, especially when they involve a real database.

Maybe you can create smaller tests (Unit tests, micro tests, ...) to achieve independence. If this is not possible, the test could check it's precondition (database empty or whatever) and wait until it's fulfilled (or until a timeout happens).

Dropping the database in BeforeAll and AfterEach might not be enough, because Jest even runs your tests in parallel: Are tests inside one file run in parallel in Jest?

Also, dropping might not be a completely synchronous and atomic operation, especially when there is some caching in the DB driver. But I don't know mongodb and it's node.js integration well enough to judge that.