Run a test until it fails

Question

In Bazel, you can re-run a test multiple times with:

bazel test --runs_per_test=<n> <target>

This is useful to reproduce conditions that lead a flaky test to fail.

The shortcoming of this approach, though, is that the tests will be run n times regardless of the results.

For tests that are flaky in very rare conditions, this means that you have to set n high, but this means you may have to scroll through a lot of text before you find the failing test's output.

Is there a built-in way in Bazel to run a test until it fails? Right now I'm using a while loop in Bash, which is good enough for my use case but not portable:

while bazel test --test_output=errors -t- <target_name>; do :; done

Answer 1

Passing --notest_keep_going will cause Bazel to exit as soon as a test fails. So, you could use --notest_keep_going in combination with an absurdly high --runs_per_test.