Xamarin Forms Unit Testing

Question

What is the approach to write unit tests for Xamarin Forms application (as opposed to Xamarin Traditional which is Xamarin.Android, Xamarin.IOS, or Xamarin.UWP)?

Can anyone provide a good explanation for Unit Tests in Xamarin.Forms vs Unit Tests in Xamarin Traditional?

A good explanation article how to implement Xamarin.Forms tests and are they even needed or should we write Unit Tests for each platform instead?

I have read a number of articles out there but I haven't found one that starts from creating the unit test project type in Visual Studio to writing and running the tests.

They mostly start somewhere in the middle discussing DI or ServiceLocator (like this one http://arteksoftware.com/unit-testing-with-xamarin-forms-dependencyservice/).

Or, on the other hand, they mix Xamarin.Forms with Xamarin.Android (or IOS) unit testing (like this one: http://www.dsibinski.pl/2017/03/unit-testing-xamarin-application/).

Or, they mix Portable vs Shared like in the case of this one http://www.alteridem.net/2015/12/21/testing-xamarin-projects-using-nunit-3/.

What I understand so far is that I could use regular Unit Test project in VS and use either MSTest or NUnit. Or, I can write platform specific unit tests for each platform.

All of this is very confusing because authors seem to mix the terms all over the place.

A detailed answer with supporting examples would be highly appreciated as I am entirely a beginner in this area.

Answer 1

I had the same question for a while and I finally settled on a strategy that has two parts consisting of unit testing of the code and then UI testing.

The unit tests for code that has no UI. For example tests for the model, services, etc. Usually I have a unit test project and the unit tests are written against the shared library.

The UI tests are specific to the OS. I have an iOS test project and an Android test project. I thought about having a single UI test project which would be Nirvana. I just do not belief it could handle all the UI nuances of each OS.

During the build I run the unit tests. If they pass then the UI tests are run. I have two sets of tests, smoke tests and deep tests, for each OS. The smoke tests are done on a small subset of devices. I can quickly judge the quality of a build before wasting test time on bad builds. Then on good builds the deep tests are performed on the same small subset of devices. If everything passes then smoke tests are performed on a larger pool of devices. If crash reports come in on a device in the large pool then the deep tests are performed. If the device in question has a large enough users base and continues to be problematic it is added to the deep pool testing.

I learned on the job over a very very long career including 7 years working at Microsoft including being a PM on a couple of teams working on the .NET Framework, .NET Framework SDK and related technologies like ASP.NET MVC, WebAPI, Azure, etc. The build process I described above, is loosely based on how the .NET Framework and .NET Framework SDK use to be built. The build for .NET takes a while. The test runs even longer (think physical days) so having a quick smoke test as part of the build was very useful and time saving. Some times the build would compile and produce installs but would have problems that a few (relatively speaking) tests could point to a bad build and save lots of time for the QA teams.

Rookiejava below has added some good links for learning how to write tests.

My best advice for designing tests is be stupid, mean, strict and a gremlin. By stupid make tests that just do things no one who understands programing would do. The best example of this was a login bug found on I believe x-box. Someone's kid did a really flakey thing and lo and behold got into his father's account. By be mean I mean just that do maddening things that you would make you mad. Pun intended. For example hitting on your product. No kidding there is a major safe manufacturer that hitting the safe in a certain place allows you to open it without the code! Literally be strict and do not make assumptions. If the use case, user story or what document you use for knowing how the code is suppose to works says fatique then your test should let the code pass only on fatique even though we all know it should be fatigue. Do file a bug against the use case/user story ;) The biggest piece of advice I can give is think like a gremlin. Don't just test that the happy path works or data validation rules work. Test for doing bizarre outrageous inputs and combination of commands, key presses etc. The best test tool I used was a Microsoft test tool for Windows Mobile. Yes that Windows. The tool would do completely random things to a app's UI. Random keypresses put information in from bottom to top or even in random orders among other things. It was fun to see how many unconsciously made assumptions I and others had made. It even found a bug that could allow someone complete access to a device I was working on for a government agency that not only our team missed but other outside security testers missed. We quietly patched it before the device ever went to anybody outside development.