Flutter integration tests for iOS on Firebase?

Question

Is it possible to run Flutter integration tests on Firebase? There seems to be conflicting info on this - some sources say its possible, but on the documentation, the iOS section seems to only be for testing on your own device.

Firebase asks for an XCT test package to be uploaded. How can it be obtained / created for a Flutter project?

Answer 1

Yes, you can write Flutter integration tests for iOS on Firebase Test Lab. Even with Dart code and not Swift code. You need to use the integration_test package. You will find in the Flutter documentation how to set up the integration_test package: https://flutter.dev/docs/testing/integration-tests

Your test will look like something like this:

import 'package:flutter/material.dart';
import 'package:flutter_test/flutter_test.dart';
import 'package:integration_test/integration_test.dart';
import '../lib/main.dart' as app;

// How to run?
// flutter drive --driver=test_driver/integration_test.dart --target=integration_test/app_test.dart

void main() {
  IntegrationTestWidgetsFlutterBinding.ensureInitialized();

  // Currently, we can't setup intergration tests, because there is a issue with integraiton test package +
  // localization.
  //
  // See this ticket for information: https://github.com/flutter/flutter/issues/84053
  testWidgets('full app test', (tester) async {
    app.main();
    await tester.pumpAndSettle();
    await tester.pump();

    // Verify that our counter starts at 0.
    expect(find.text('0'), findsOneWidget);
    expect(find.text('1'), findsNothing);

    // Tap the '+' icon and trigger a frame.
    await tester.tap(find.byIcon(Icons.add));
    await tester.pump();

    // Verify that our counter has incremented.
    expect(find.text('0'), findsNothing);
    expect(find.text('1'), findsOneWidget);
  });
}

To be able to run this test as a XCTest (this is what Firebase Test Lab needs), you need to some extra steps:

Open ios/Runner.xcworkspace in Xcode. Create a test target if you do not already have one via File > New > Target... and select Unit Testing Bundle. Change the Product Name to RunnerTests. Make sure Target to be Tested is set to Runner and language is set to Objective-C. Select Finish. Make sure that the iOS Deployment Target of RunnerTests within the Build Settings section is the same as Runner.

Add the new test target to ios/Podfile by embedding in the existing Runner target.

target 'Runner' do
  # Do not change existing lines.
  ...

  target 'RunnerTests' do
    inherit! :search_paths
  end
end

To build integration_test/foo_test.dart from the command line, run:

flutter build ios --config-only integration_test/foo_test.dart

In Xcode, add a test file called RunnerTests.m (or any name of your choice) to the new target and replace the file:

@import XCTest;
@import integration_test;

INTEGRATION_TEST_IOS_RUNNER(RunnerTests)

Run Product > Test to run the integration tests on your selected device.

To deploy it to Firebase Test Lab you can follow these steps:

Execute this script at the root of your Flutter app:

output="../build/ios_integ"
product="build/ios_integ/Build/Products"
dev_target="14.3"

# Pass --simulator if building for the simulator.
flutter build ios integration_test/foo_test.dart --release

pushd ios
xcodebuild -workspace Runner.xcworkspace -scheme Runner -config Flutter/Release.xcconfig -derivedDataPath $output -sdk iphoneos build-for-testing
popd

pushd $product
zip -r "ios_tests.zip" "Release-iphoneos" "Runner_iphoneos$dev_target-arm64.xctestrun"
popd

You can verify locally that your tests are successful by running the following command:

xcodebuild test-without-building -xctestrun "build/ios_integ/Build/Products/Runner_iphoneos14.3-arm64.xctestrun" -destination id=<YOUR_DEVICE_ID>

Once everything is ok, you can upload the resulting zip to Firebase Test Lab (change the model with your values):

gcloud firebase test ios run --test "build/ios_integ/ios_tests.zip" --device model=iphone11pro,version=14.1,locale=fr_FR,orientation=portrait

Note: I copied to the needed extra for iOS device tests steps from https://github.com/flutter/flutter/tree/master/packages/integration_test#ios-device-testing (credits to the Flutter team ❤️)

If you want to run these tests on Android Firebase Test Lab, you need also to do some extra steps. You will find these steps under: https://github.com/flutter/flutter/tree/master/packages/integration_test#android-device-testing. I can also recommend to check out this video, where Reso Coder explained and showed every little step: https://www.youtube.com/watch?v=izajHHFSa8o

Answer 2

First of all open your ios Module in Xcode as ss below.

Now you are free to write cool XCode UI tests with Swift. For example I am dropping here a Firebase Login test for your app maybe you wish to use.

import XCTest
import UIKit
import Firebase

class LoginTestUITests: XCTestCase {

    var app: XCUIApplication!
    
    func testLogin() {
        continueAfterFailure = false
        app = XCUIApplication()
        app.launch()
        wait(for: 2)
        passLogin()
        wait(for: 5)
     // after your login you can add some methods here according to your app flow
    }
}

extension LoginTestUITests {

func passLogin() {
    
    //put your logic here        
    
}
// this is a pretty cool extension waiting according to input for a while coming the data from backend etc.
extension XCTestCase {
    func wait(for duration: TimeInterval) {
        let waitExpectation = expectation(description: "Waiting")
        
        let when = DispatchTime.now() + duration
        DispatchQueue.main.asyncAfter(deadline: when) {
            waitExpectation.fulfill()
        }
        // I use a buffer here to avoid flakiness with Timer on CI
        waitForExpectations(timeout: duration + 0.5)
    }
}

Answer 3

Open Xcode project (by default, it's ios/Runner.xcodeproj). Create a test target (navigating File > New > Target... and set up the values) and a test file RunnerTests.m and change the code. You can change RunnerTests.m to the name of your choice.

#import <XCTest/XCTest.h>
#import <integration_test/IntegrationTestIosTest.h>

INTEGRATION_TEST_IOS_RUNNER(RunnerTests)