Mutation testing in cpp without modifying/recompiling the code

Question

Currently I have some tests that are blamed to not catch bugs very well. I want to do mutation testing in order to detect them (and prevent from adding new useless ones), but without the time-inefficient loop: change the code -> recompile -> run tests -> change the code -> recompile -> run tests... etc.

Initially I wanted to mutate somehow binary elf files directly (no recompilation), but as later posts suggested, it makes no sense.

Answer 1

Ok, I was able to solve it partially, by splitting the mutation testing into 4 main stages:

• instrument the code in all mutations with python/clang-tooling (selected C++ expressions are wrapped in a special macro, that delegates calls to a mutation class, which generates ID for each mutation, controls activation of mutation operators, etc.)
• recompile the code (only once)
• run tests in parallel, with all mutations inactive, and obtain IDs of all mutations (if there are failing tests, put them on an ignore list),
• run tests in parallel while switching mutations in the run-time (by ID obtained in the previous step), and gather statistics (mutant kill ratio, etc.)

The implementation is done in python and C++, it is around ~1700 lines of code (with tests) + minor adaptations in the production code (CMake, and the gtest main.cpp file). It supports only a couple of simple mutations, but it still makes fun :)