Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

async/await unit test code coverage

How to write unit test for async/await method, I am using Visual Studio 2013.

Suppose we have a async method:

public async Task DoSomethingAsync()
{
    ...
    await _service.DoInternalAsync();
    ...
}

Since I am using the latest version of Visual Studio, it has good support to the async method unit test:

[TestMethod]
public async Task DoSomthingAsyncTest()
{
    ...
    await _objectUnderTest.DoSomethingAsync();
    // how to verify the result??? here is what I did
    _service.Verify(_ => _.DoInternalAsync());
}

Basically, I have two questions:

  1. As commented in the code, how to verify the Task result? Did I do that right way?
  2. If I run that test, the VS would say test passed. But when I check the code coverage, the await _service.DoInternalAsync() sentence seems not be covered, from the view of Code Coverage Results, it prompt the MoveNext() sentence has 6 not covered blocks. What's wrong within it?
like image 720
Jerry Bian Avatar asked Oct 15 '14 03:10

Jerry Bian


People also ask

How do you calculate unit testing code coverage?

To calculate the code coverage percentage, simply use the following formula: Code Coverage Percentage = (Number of lines of code executed by a testing algorithm/Total number of lines of code in a system component) * 100.

What is acceptable unit test coverage?

With that being said it is generally accepted that 80% coverage is a good goal to aim for. Trying to reach a higher coverage might turn out to be costly, while not necessary producing enough benefit. The first time you run your coverage tool you might find that you have a fairly low percentage of coverage.

How do you run tests with coverage?

Run a test with code coverageOpen the desired file in the Project tool window and choose Run <name> with Coverage from the context menu. You can also select a directory with test files and choose the corresponding command from the context menu to run several tests with coverage.

Is code coverage the same as test coverage?

While code coverage helps you verify if each code in the software application is being executed by existing tests or not, test coverage indicates whether those tests are covering all the functional requirements of the application or not.


2 Answers

Okay, from my research, the code coverage issue is a Visual Studio bug in latest version Visual Studio 2013, they would fix/enhance it in next major version.

Quote from the feedback:

The issue that you are seeing is due to a bug from our end due to which we do not have complete support for the async/await pattern in code coverage as yet. The work is pending and should be something we deliver in the next major update/release. There are no clean workarounds for this issue.

like image 181
Jerry Bian Avatar answered Oct 27 '22 18:10

Jerry Bian


The reason the code is not shown as being covered has to do with how async methods are implemented. The C# compiler actually translates the code in async methods into a class that implements a state machine, and transforms the original method into a stub that initialized and invokes that state machine. Since this code is generated in your assembly, it is included in the code coverage analysis.

If you use a task that is not complete at the time the code being covered is executing, the compiler-generated state machine hooks up a completion callback to resume when the task completes. This more completely exercises the state machine code, and results in complete code coverage (at least for statement-level code coverage tools).

A common way to get a task that is not complete at the moment, but will complete at some point is to use Task.Delay in your unit test. However, that is generally a poor option because the time delay is either too small (and results in unpredictable code coverage because sometimes the task is complete before the code being tests runs) or too large (unnecessarily slowing the tests down).

A better option is to use "await Task.Yield()". This will return immediately but invoke the continuation as soon as it is set.

Another option - though somewhat absurd - is to implement your own awaitable pattern that has the semantics of reporting incomplete until a continuation callback is hooked up, and then to immediately complete. This basically forces the state machine into the async path, providing the complete coverage.

To be sure, this is not a perfect solution. The most unfortunate aspect is that it requires modification to production code to address a limitation of a tool. I would much prefer that the code coverage tool ignore the portions of the async state machine that are generated by the compiler. But until that happens, there aren’t many options if you really want to try to get complete code coverage.

A more complete explanation of this hack can be found here: http://blogs.msdn.com/b/dwayneneed/archive/2014/11/17/code-coverage-with-async-await.aspx

like image 44
Dwayne Need Avatar answered Oct 27 '22 20:10

Dwayne Need