Visual Studio Unit Tests running slower on TFS Build

Question

My project has 1000+ unit tests that, in a local machine, all run in less than 10 seconds. But when they run on TFS Build, some tests run significantly slower than others. 3 of them run in about 1-2 minutes, other 4 in 5-30 seconds, and the others in fractions of a second. I've noticed that all those slower tests use fakes from Microsoft Fakes, and each one of them is the first to run in it's class. But a lot of the other tests also use fakes (some more intensively) and run in regular time. I would like to know what may be causing that slowdown and how can I fix it.

Edit: I've noticed that every slower test runs after a mockless test. Maybe that slowdown is caused by the initialization of the ShimsContext. In my test classes, the ShimsContext is created and disposed on TestInitialize and TestCleanup methods. Does that affects significantly the performance?

Answer 1

First off all I would strongly recommend you move away from shims. They are a crutch, and aside from a very few scenarios, simply not needed. Design your code for testability, and you will find you can do without them.

Second, shims are not thread safe, and cannot be used concurrently safely. Likely this is why you are seeing the really slow run times.

Either your local is running things concurrently when it shouldn't (MS says it isn't safe, but not enforced), and the build server isn't.

Or the build server is trying to be parallel and it is causing issues.

Tweak your concurrency settings to disable it and see what effect that has on your runtime.