Timed out waiting for result of ClearAppData2 when running Xamarin UI Tests for Android

Question

I've created automatic Android and iOS UI-Tests for my Xamarin application with the Xamarin UITest framework. When running the tests locally, they work fine, but when running them on the Bitrise CI, the iOS tests run fine, but the Android UI Tests keep failing with the following exception:

StartFirstActivity_WaitForActivity_ExpectButtonToHaveText
SetUp : System.Exception : Timed out waiting for result of ClearAppData2
Stack trace:
  at Xamarin.UITest.Shared.Android.Commands.CommandAdbClearAppData.Execute (IProcessRunner processRunner, IAndroidSdkTools androidSdkTools) <0x38b3e90 + 0x0064b> in <filename unknown>:0 
  at Xamarin.UITest.Shared.Execution.Executor.Execute[TDep1,TDep2] (ICommand2 command) <0x32b6478 + 0x00092> in <filename unknown>:0 
  at Xamarin.UITest.Shared.Android.LocalAndroidAppLifeCycle.EnsureInstalled (Xamarin.UITest.Shared.Android.ApkFile appApkFile, Xamarin.UITest.Shared.Android.ApkFile testServerApkFile) <0x37418c8 + 0x0017a> in <filename unknown>:0 
  at Xamarin.UITest.Android.AndroidApp..ctor (IAndroidAppConfiguration appConfiguration) <0x31a15e8 + 0x0047a> in <filename unknown>:0 
  at Xamarin.UITest.Configuration.AndroidAppConfigurator.StartApp (AppDataMode appDataMode) <0x30b4298 + 0x00063> in <filename unknown>:0 
  at SightPlayer.Core.Test.AppInitializer.StartApp (Platform platform) <0x30b2448 + 0x000ef> in <filename unknown>:0 
  at SightPlayer.Core.Test.Tests.BeforeEachTest () <0x30b23f8 + 0x00013> in <filename unknown>:0 
  at (wrapper managed-to-native) System.Reflection.MonoMethod:InternalInvoke (System.Reflection.MonoMethod,object,object[],System.Exception&)
  at System.Reflection.MonoMethod.Invoke (System.Object obj, BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) <0x30b2208 + 0x00093> in <filename unknown>:0

Android test are running with Xamarin.UITest version 1.3.5. This is important, as there seems to have been a bug until version 1.3.3. I've also tried ignoring failing tests, but then the test-runner fails with other Android tests. Interestingly enough that sometimes individual tests pass.

Has anyone encountered this behaviour before? Do you have any advice on how to fix this?

When decompiling CommandAdbClearAppData (which throws the exception), I see

// ISSUE: reference to a compiler-generated field
string str1 = string.Format("/data/data/{0}/files/calabash_failure.out", (object)executeCAnonStorey0.svr);
// ISSUE: reference to a compiler-generated field
string str2 = string.Format("/data/data/{0}/files/calabash_finished.out", (object)executeCAnonStorey0.svr);
while (DateTime.UtcNow < utcNow + TimeSpan.FromSeconds(10.0))
{
    if (func(string.Format("ls {0}", (object)str1)).Output.Trim().Equals(str1))
        throw new Exception("Clear app data failed with " + func(string.Format("cat {0}", (object)str1)).Output);
    if (func(string.Format("ls {0}", (object)str2)).Output.Trim().Equals(str2) && func(string.Format("cat {0}", (object)str2)).Output.Trim().Equals("SUCCESSFUL"))
        return;
}
throw new Exception("Timed out waiting for result of ClearAppData2");

which indicates that the generated file could not be found within ten seconds. Could it be, that the emulator is simply too slow and the emulator takes longer than ten seconds to create those files?

Answer 1

Could it be, that the emulator is simply too slow and the emulator takes longer than ten seconds to create those files?

Yes, that could be the problem.

Do you have any advice on how to fix this?

The latest dev builds of the Xamarin.UITest nuget package have increased that timeout interval from 10 seconds to 60 seconds.

Try Xamarin.UITest 1.3.6.1476-dev or newer.

Answer 2

I solved this issue uploading a unsigned version of the APK to my testing device (same thing for Xamarin Android Player). I had to uncheck the "Sign the .APK file" box from the Android Package Signing option in the project properties. The WaitTimes way didn't work for me.

Answer 3

I think you need to look at some Configuration

If your app is not waiting to launch the test you could implement WaitTimes.

Create a class like this:

public class WaitTimes : IWaitTimes
{
    public TimeSpan GestureWaitTimeout
    {
        get { return TimeSpan.FromMinutes(1); }
    }

    public TimeSpan WaitForTimeout
    {
        get { return TimeSpan.FromMinutes(1); }
    }
}

Then implement it like this:

_app = ConfigureApp 
    .Android
    .EnableLocalScreenshots()
    .ApkFile(apkFile)
    .DeviceSerial("###")
    .ApiKey("###")
    .Debug()
    .WaitTimes(new WaitTimes()) 
    .StartApp();

Otherwise you could just wait for a specific element to load, like this:

_app.WaitForElement(c => c.Marked("Login"), "Time out expired", TimeSpan.FromSeconds(15));
var result = _app.Query(c => c.Marked("Login"));
Assert.AreEqual(1,result.Length);
_app.Screenshot("Test passed with success");

Here is some more information about WaitTime and WaitForElement that could be helpful.