Menu
I have 2 Unit-Test-Assemblies:
both make use of the Microsoft Fakes Framework (using Stubs and Shims).
Running the unit tests locally works fine (tested on 4 different machines, even on the Visual Studio installed on the build server), however if we build using a build agent, some unit test fails with the exception, e.g.:
Unable to create instance of class UserTests.ClientUserTest. Error: System.TypeLoadException: Could not load type 'WorldDirect.CCM.Shared.Backend.SmartClassic.Fakes.StubClient' from assembly 'WorldDirect.Smart.Backend.Fakes, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null'.
or the same for a shim:
Test method ExtensionTests.ExtensionTests.UpdateExtensionValidate_NoGrnp_ChecksIpPbxDependencies threw exception: System.TypeLoadExceptio: Could not load type 'WorldDirect.CCM.Shared.Backend.SmartClassic.Fakes.ShimIpPbxRemoteDestination' from assembly 'WorldDirect.Smart.Backend.Fakes, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null'.
The interesting thing is, they fail sporadically, I've tried to make up a little statistics:
# | Outcome | Comment
19 | 64/64 passed | only 'UserTests'-UnitTests run, 'ExtensionTests' inactive
20 | 37/64 passed | same source code as #19
21 | 64/64 passed | same source code as #19
22 | 64/66 passed | all 'UserTests' run as well as 2 'ExtensionTests', others still inactive; the 2 ExtensionTests failed with ShimIpPbxRemoteDestination-TypeLoadException
23 | 38/65 passed | same source code as #22, notice NOTHING has changed, however 1 unit test was not even run in 'ExtensionTests'; however the second succeeds; all unit tests in 'UserTests' using MS Fakes fail with a StubClient-TypeLoadException
24 | 38/65 passed | same source code as #22
new day - no more luck :-(
1 | 37/64 passed | uncommented the 2 'ExtensionTests', so same code as #19
2 | 37/64 passed | so same code as #1
3 | 64/64 passed | so same code as #1; suddenly they all work again
Notice that beside the mentioned changes nobody has changed the source code.
All build were made using a detailed log-level ;
Shim Diagnostic="true";
only explicitly needed shims/stubs are included in .fakes-file (otherwhise MSBUILD was failing using exit-code 1).
It all seems quite magic to me, possibly somebody has already experienced the same problems or somebody has a hint.
Thanks in advance
823
Add Fakes AssemblyIn Solution Explorer, open your unit test project's references and select the reference to the assembly that contains the method you want to fake. In this example, the DateTime class is in System. dll. To see the references in a Visual Basic project, select Show All Files.
Shim, in C#, is a template class that is derived from a base class with derived classes that inherit the data and behavior of the base class and vary only in the type. The derived class of the shim class reuses the implementation provided by the shim class.
I think I know where the issue lies, the TypeNotFoundException was correct, at least the corresponding types were not generated (as shown by Reflector), this made me thinking. More exactly there is only one generated fake-file generated per assembly lying under [buildName]\Binaries- and as both of my Assemblies faked the same Assembly (backend.dll) - this resulted most probably in a race condition - the one assembly which was built later than the other had its final shims/stubs generated; this doesn't clarify all problems though.
Anyway the reason why this doesn't show up on a regular VS build is because the assemblies are generated in their respective [assembly]\FakeAssemblies directory, not influencing any other unit test (the way it is supposed to be).
So my idea to circumvent this problem was to put ALL shims/stubs used by both test-assemblies into both .fakes-configuration-files; reflector showed that really all needed shims/stubs got generated this time, however the build failed with
Exception Message: MSBuild error 1 has ended this build.
With no more information in any (detailed) log. This problem is caused by not using a generated shim/stub-object specified in the .fakes-file (and of course this is also a problem just shown by the build server, would be too boring otherwise ;-)).
So the final solution I came up with is to put ALL code that makes use of shims/stubs into the same assembly, where only 1 .fakes-file configures the fake-Assembly.
Now it is working fine :-), but I think the MS-Dev-Team has still something to do on here, haven't tested it with TFS2013 though; gonna submit a bug anyway (as I've discovered 2 already)
Hope this helps somebody with the same problem.
135
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With