I have some C# performance tests, basically running two different methods and checking that one runs much faster than the other.
When I run them locally in NUnit, one of the tests runs ten times as fast as the other, so I've got an NUnit test that uses Stopwatch
to check that it is at least twice as fast (in case of regression). But when I run the tests in TeamCity, the fast method is only about 1.5 times as fast as the slow one? I would expect hardware differences to have some effect, but not this much. What could be causing this?
To answer my own question, the problem turned out to be that code coverage was turned on for the test build in TeamCity, so the overhead of this brought the two method runtimes closer together. Hopefully this answer helps someone else in future.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With