Is it possible to use Catch2 for testing an MPI code?

Question

I'm working with a fairly big MPI code. I started to include unit tests into the existing code base. However, as soon as the unit under test uses an MPI routine the test executable crashes with the error message "calling an MPI routine before MPI_Init"

• What the best way around this?
• Can I run tests with multiple MPI ranks?

Answer 1

Yes, it is possible.

As explained in https://github.com/catchorg/Catch2/issues/566, you will have to supply a custom main function.

#define CATCH_CONFIG_RUNNER
#include "catch.hpp"
#include <mpi.h>

int main( int argc, char* argv[] ) {
    MPI_Init(&argc, &argv);
    int result = Catch::Session().run( argc, argv );
    MPI_Finalize();
    return result;
}

To amplify your experience using Catch2 in combination with MPI, you might want to avoid redundant console output. That requires injecting some code into ConsoleReporter::testRunEnded of catch.hpp.

#include <mpi.h>
void ConsoleReporter::testRunEnded(TestRunStats const& _testRunStats) {
    int rank id = -1;
    MPI Comm rank(MPI COMM WORLD,&rank id);
    if(rank id != 0 && testRunStats.totals.testCases.allPassed())
        return;
    printTotalsDivider(_testRunStats.totals);
    printTotals(_testRunStats.totals);
    stream << std::endl;
    StreamingReporterBase::testRunEnded(_testRunStats);
}

Finally you might also want to execute your test cases with a different number of MPI ranks. I found the following to be an easy and well working solution:

SCENARIO("Sequential Testing", "[1rank]") {
    // Perform sequential tests here
}
SCENARIO("Parallel Testing", "[2ranks]") {
    // Perform parallel tests here
}

Then you can call the tagges scenarios individually with

mpiexec -1 ./application [1rank]
mpiexec -2 ./application [2rank]

Answer 2

For anyone looking to remove all duplicated console output when running Catch2 distributed, here's a solution.

Find the definition of ConsoleReporter in catch.hpp (in v2.10.0, it's at line 15896). It will look something like:

ConsoleReporter::ConsoleReporter(ReporterConfig const& config)
    : StreamingReporterBase(config),
    m_tablePrinter(new TablePrinter(config.stream(),
        [&config]() -> std::vector<ColumnInfo> {
        if (config.fullConfig()->benchmarkNoAnalysis())
        {
            return{
                { "benchmark name", CATCH_CONFIG_CONSOLE_WIDTH - 43, ColumnInfo::Left },
                { "     samples", 14, ColumnInfo::Right },
                { "  iterations", 14, ColumnInfo::Right },
                { "        mean", 14, ColumnInfo::Right }
            };
        }
        else
        {
            return{
                { "benchmark name", CATCH_CONFIG_CONSOLE_WIDTH - 32, ColumnInfo::Left },
                { "samples      mean       std dev", 14, ColumnInfo::Right },
                { "iterations   low mean   low std dev", 14, ColumnInfo::Right },
                { "estimated    high mean  high std dev", 14, ColumnInfo::Right }
            };
        }
    }())) {}
ConsoleReporter::~ConsoleReporter() = default;

Though not shown here, the base-class StreamingReporterBase provides a stream attribute which we'll disable, by the failbit trick shown here.

Inside the final {} above (an empty constructor definition), insert:

// I solemnly swear that I am up to no good
int rank;
MPI_Comm_rank(MPI_COMM_WORLD, &rank);

// silence non-root nodes
if (rank != 0)
    stream.setstate(std::ios_base::failbit);  

You can see an example on this repo.