Can we get native C++ code coverage in VS2012 or VS2010 without MSTest?

Question

We would like to measure code coverage for our own automated regression test system run over a fairly large native app. This is a sophisticated, scripted test system using the inbuilt scripting of our app. It has thousands of tests and is not going to be replaced by MSTest unit tests.

Whilst we're using VS2012 (Premium) as the IDE currently it is still compiled with the VS2010 compilers & libraries. That could change sooner if it was a prerequisite to getting code coverage going.

We can do separate builds for this - instrumenting is not a problem.

I'm just confused reading the MS documentation which seems to all start from an assumption you're running unit tests using their inbuilt test framework. That's when I'm not struggling to find stuff which actually talks about native support for ALM in the first place!

thanks

Answer 1

Visual Studio 2012's code coverage tool is entirely separate from the test execution system (full disclosure: I wrote it, but the team that inherited it after I left Microsoft removed some fairly useful functionality). It was rewritten from the ground up in VS 2012 to dynamically instrument native (x86 and x86-64) and managed code (.NET and Silverlight) when it loads into the process instead of modifying executables on disk.

You can find CodeCoverage.exe in "%ProgramFiles%\Microsoft Visual Studio 11.0\Team Tools\Dynamic Code Coverage Tools".

To collect data:

CodeCoverage.exe collect /output:foo.coverage foo.exe foos_args

A configuration file (there's a default one in that directory called CodeCoverage.config) can be specified to control collection.

To analyze the coverage data, you can open foo.coverage in Visual Studio 2012 or use the coverage tool itself to do the analysis:

CodeCoverage.exe analyze /output:results.xml foo.coverage

Note: for instrumentation to take place, .pdb files must be discovered for your modules. Since you are building with 2010, they may not work with 2012's DIA so you may have to rebuild with 2012's toolset. If you are not seeing the modules you expect in the coverage analysis, pass /include_skipped_modules to the analyze command; there will be a "reason" attribute telling you why the module was skipped (excluded, no debug information, etc.).

Edit: Also, unlike previous versions of Visual Studio, 2012's coverage file format is completely self-contained. The modules and .pdbs don't need to be present at analysis time.

Answer 2

I realize this is an old post, but I believe the answer still is relevant.

With all the things that I used to have at my disposal in C#, I didn't really like what I saw when I moved to Visual C++. Also, like you the MSTests only partially worked for me; I'm used to have my own test applications as well.

Basically what I wanted was the following:

• Run MS tests or an EXE file
• Get code coverage right in Visual Studio.

After doing some research, I noticed that VS Enterprise supports this feature today with test adapters.

If you're not on VSE, I noticed there are some other tools, each providing users with an independent UI. Personally I don't like that; I want my coverage right in Visual Studio, preferably in Visual Studio Community edition.

So I decided to build this addin myself and - while it's not as sophisticated as VSE - it does the trick for me.

• I wrote a VSIX code coverage tool on https://github.com/atlaste/CPPCoverage . Basically it manages the highlighting in Visual Studio, generates a clickable report, and integrates in solution explorer.
• For the coverage measurements themselves, I used to use https://opencppcoverage.codeplex.com/ . Basically that allows you to perform code coverage tests on any debuggable (native) executable. Nowadays, I'm using my own code coverage measuring tools (which are open sourced above as well).