Testing my Xamarin.Forms iOS Application with Xamarin.UITest: The first test always fails

Question

I am testing an Xamarin.iOS Application on a real device (iPhone 6s with iOS 12.1) with Xamarin.UITest. When I am running my few UI Tests, the first test always seam to crash (regardless of what is happening inside the test) due to the same error (see below).

Enviroment is:

• Xamarin.UITest 2.2.7

• NUnitTestAdapter 2.1.1

• NUnit 2.6.3

• Xamarin.TestCloud.Agent 0.21.7

Setup is:

[SetUp]
public void Setup(){
     this.app = ConfigureApp.iOS
        .EnableLocalScreenshots()
                .InstalledApp("my.app.bundle")
                .DeviceIdentifier("My-iPhone6s-UDID-With-iOS12.1")  
                .StartApp();
}

[Test]
public void FirstTestThatCouldBeEmpty(){
     //But doesn't have to be empty to produce the error    
}

Resulting error:

2019-01-17T14:11:20.4902700Z 1 - ClearData:> 2019-01-17T14:11:20.4916340Z bundleId: my.app.bundle 2019-01-17T14:11:20.4929580Z deviceId: My-iPhone6s-UDID-With-iOS12.1

2019-01-17T14:11:33.7561260Z 3 - LaunchTestAsync:

2019-01-17T14:11:33.7574050Z deviceId: My-iPhone6s-UDID-With-iOS12.1

2019-01-17T14:11:33.9279420Z 5 - HTTP request failed, retry limit hit

2019-01-17T14:11:33.9302300Z Exception: System.Net.Http.HttpRequestException: An error occurred while sending the request ---> System.Net.WebException: Unable to read data from the transport connection: Connection reset by peer. ---> System.IO.IOException: Unable to read data from the transport connection: Connection reset by peer. ---> System.Net.Sockets.SocketException: Connection reset by peer

2019-01-17T14:11:33.9322710Z at System.Net.Sockets.Socket.EndReceive (System.IAsyncResult asyncResult) [0x00012] in <23340a11bb41423aa895298bf881ed68>:0

2019-01-17T14:11:33.9340560Z at System.Net.Sockets.NetworkStream.EndRead (System.IAsyncResult asyncResult) [0x00057] in <23340a11bb41423aa895298bf881ed68>:0

2019-01-17T14:11:33.9358740Z --- End of inner exception stack trace ---

2019-01-17T14:11:33.9377100Z at System.Net.Sockets.NetworkStream.EndRead (System.IAsyncResult asyncResult) [0x0009b] in <23340a11bb41423aa895298bf881ed68>:0

2019-01-17T14:11:33.9398100Z at System.IO.Stream+<>c.b__43_1 (System.IO.Stream stream, System.IAsyncResult asyncResult) [0x00000] in <98fac219bd4e453693d76fda7bd96ab0>:0

2019-01-17T14:11:33.9415720Z at System.Threading.Tasks.TaskFactory1+FromAsyncTrimPromise1[TResult,TInstance].Complete (TInstance thisRef, System.Func`3[T1,T2,TResult] endMethod, System.IAsyncResult asyncResult, System.Boolean requiresSynchronization) [0x00000] in <98fac219bd4e453693d76fda7bd96ab0>:0

Sometimes it is this error:

SetUp : Xamarin.UITest.XDB.Exceptions.DeviceAgentException : Unable to end session: An error occurred while sending the request

System.Net.Http.HttpRequestException : An error occurred while sending the request

System.Net.WebException : Unable to read data from the transport connection: Connection reset by peer.

System.IO.IOException : Unable to read data from the transport connection: Connection reset by peer.

System.Net.Sockets.SocketException : Connection reset by peer

What could be a solution to this? Doing the version puzzle of the nuget packages got me this far, that all tests are working besides this one.

A dummy test would be a possibility, but that would that countinous builds with those test would never be in the state of "successful" because this one test that is failing =(

Answer 1

I was able to find a work around for this issue, and really any issue with the device agent not connecting to the app under test.

        public static IApp StartApp(Platform platform)
        {
            if (platform == Platform.iOS)
            {
                try
                {
                    return ConfigureApp.iOS
                       .InstalledApp("com.example")
#if DEBUG
                       .Debug()
#endif
                       .StartApp();
                }
                catch
                {
                    return ConfigureApp.iOS
                       .InstalledApp("com.example")
#if DEBUG
                        .Debug()
#endif
                       .ConnectToApp();
                }
            }

In plain language : "If you tried to start the app, and something went wrong, try to see if the app under test was started anyway and connect to it"

Note : As https://stackoverflow.com/users/1214857/iupchris10 mentioned in the comments, a caveat to this approach is that when you run tests consecutively, while the StartApp call will in theory reliably shut down the app, with the point of failure being reconnection, if it does not and you are running tests consecutively, you will simply connect to the running app instance in whatever state it was left in. This approach offers no recourse. Xamarin's StartApp will throw Exception, rather than a typed exception, and so mitigating this will rely on you parsing InnerException.Message and keeping a table of expected failures.