Are the built-in integration tests run concurrently or sequentially?

Question

I am writing integration tests to work with a database. In the start of each test, I clear the storage and create some data.

I want my tests to run sequentially to ensure that I am working with an empty database. But it seems that integration tests are run concurrently because sometimes I get existing documents after cleaning the database.

I checked the database and found that the documents created in different tests have approximately the same creation time, even when I'm adding a delay for each test (with std::thread::sleep_ms(10000)).

Can you clarify how the integration tests are run and is it possible run them in order?

Answer 1

The built-in testing framework runs concurrently by default. It is designed to offer useful but simple support for testing, that covers many needs, and a lot of functionality can/should be tested with each test independent of the others. (Being independent means they can be run in parallel.)

That said, it does listen to the RUST_TEST_THREADS environment variable, e.g. RUST_TEST_THREADS=1 cargo test will run tests on a single thread. However, if you want this functionality for your tests always, you may be interested in not using #[test], or, at least, not directly.

The most flexible way is via cargo's support for tests that completely define their own framework, via something like the following in your Cargo.toml:

[[test]]
name = "foo"
harness = false

With that, cargo test will compile and run tests/foo.rs as a binary. This can then ensure that operations are sequenced/reset appropriately.

Alternatively, maybe a framework like stainless has the functionality you need. (I've not used it so I'm not sure.)

Answer 2

An alternative to an env var is the --test-threads flag. Set it to a single thread to run your tests sequentially.

cargo test -- --test-threads 1