Passing custom command-line arguments to a Rust test

Question

I have a Rust test which delegates to a C++ test suite using doctest and wants to pass command-line parameters to it. My first attempt was

// in mod ffi
pub fn run_tests(cli_args: &mut [String]) -> bool;

#[test]
fn run_cpp_test_suite() {
    let mut cli_args: Vec<String> = env::args().collect();
    if !ffi::run_tests(
        cli_args.as_mut_slice(),
    ) {
        panic!("C++ test suite reported errors");
    }
}

Because cargo test help shows

USAGE:
    cargo.exe test [OPTIONS] [TESTNAME] [-- <args>...]

I expected

cargo test -- --test-case="X"

to let run_cpp_test_suite access and pass on the --test-case="X" parameter. But it doesn't; I get error: Unrecognized option: 'test-case' and cargo test -- --help shows it has a fixed set of options

Usage: --help [OPTIONS] [FILTER]

Options:
        --include-ignored 
                        Run ignored and not ignored tests
        --ignored       Run only ignored tests
...

My other idea was to pass the arguments in an environment variable, that is

DOCTEST_ARGS="--test-case='X'" cargo test

but then I need to somehow split that string into arguments (handling at least spaces and quotes correctly) either in Rust or in C++.

Answer 1

There are two pieces of Rust toolchain involved when you run cargo test.

cargo test itself looks for all testable targets in your package or workspace, builds them with cfg(test), and runs those binaries. cargo test processes the arguments to the left of the --, and the arguments to the right are passed to the binary.

Then,

Tests are built with the --test option to rustc which creates an executable with a main function that automatically runs all functions annotated with the #[test] attribute in multiple threads. #[bench] annotated functions will also be run with one iteration to verify that they are functional.

The libtest harness may be disabled by setting harness = false in the target manifest settings, in which case your code will need to provide its own main function to handle running tests.

The “libtest harness” is what rejects your extra arguments. In your case, since you're intending to run an entire other test suite, I believe it would be appropriate to disable the harness.

1. Move your delegation code to its own file, conventionally located in tests/ in your package directory:

   Cargo.toml
   src/
       lib.rs
       ...
   tests/
       cpp_test.rs
   

2. Write an explicit target section in your Cargo.toml for it, with harness disabled:

   [[test]]
   name = "cpp_test"
   # path = "tests/cpp_test.rs"   # This is automatic; you can use a different path if you really want to.
   harness = false
   

3. In cpp_test.rs, instead of writing a function with the #[test] attribute, write a normal main function which reads env::args() and calls the C++ tests.

[Disclaimer: I'm familiar with these mechanisms because I've used Criterion benchmarking (which similarly requires disabling the default harness) but I haven't actually written a test with custom arguments the way you're looking for. So, some details might be wrong. Please let me know if anything needs correcting.]

Answer 2

In addition to Kevin Reid's answer, if you don't want to write your own test harness, you can use the shell-words crate to split an environment variable into individual arguments following shell rules:

let args = var ("DOCTEST_ARGS").unwrap_or_else (|_| String::new());
let args = shell_words::split (&args).expect ("failed to parse DOCTEST_ARGS");

Command::new ("cpptest")
    .args (args)
    .spawn()
    .expect ("failed to start subprocess")
    .wait()
    .expect ("failed to wait for subprocess");