Flutter: Inline Test Coverage in Android Studio

Question

I'm developing an open source Android app in Flutter, using Android Studio 3.3.1 (on Window and Ubuntu). The source is available on Github.

The project has a test file that generates coverage data, which can be viewed with tools such as Coveralls. This indicates to me that the LCOV data contains meaningful data.

I want to use inline Code Coverage viewing, similar to the other Jetbrains tools. The run configuration under the 'Flutter Test' category correctly recoginezes my tests, and is able to run them properly.

However, the 'Run with Coverage' option is disabled. I tried different run configurations such as Android JUnit to no avail.

I am aware that I can manually create the coverage data, but my goal is to automate the generation of the coverage data, and showing the coverage inline (just like Coveralls does).

Does anyone know what run configuration, if any, accomplishes this goal?

As a side note, I recently switched to Codemagic as my CI tool so the coverage data on Coveralls is outdated, but the point that the LCOV data is meaningful still holds. I also tried similar setups in Intellij, but the result is the same as Android Studio.

Answer 1

I don't think is supported for Flutter projects yet.

I have all non-UI code in another pure Dart package that I add as dependency to the Flutter project.

For my project this also has the advantage that code that I can share with the browser GUI (Angular Dart) is separated and can't accidentally be polluted with Flutter dependencies that would break the web project.

In this project I can get coverage information in IntellJ when I follow these steps:

You need a "Dart Command Line App" IntelliJ run configuration instead of a "Dart Test", "Flutter" or "Flutter Test" run configuration.

To be able to run tests with a "Dart Command Line App" run configuration you probably need the standalone Dart SDK installed and select it in Preferences > Languages & Frameworks > Dart > Dart SDK path.

To run all tests with coverage instead of individual files you need a file like

test/all.dart

// ignore_for_file: await_only_futures

import 'dart:async';

import 'client/controller/app_controller_test.dart' as i0;
import 'client/controller/authentication_controller_test.dart' as i1;
import 'client/controller/backoffice/backoffice_controller_test.dart' as i2;
import 'client/controller/backoffice/image_reference_controller_test.dart'
    as i3;
...

Future<void> main() async {
  i0.main();
  i1.main();
  i2.main();
...
} 

with an entry for each test file.

I use a Grinder task like below to generate that file automatically

import 'package:path/path.dart' as path;
...
/// Generate a single Dart file that executes all tests.
/// Dart code coverage reporting still requires that.
@Task('generate test/all.dart')
Future<void> prepareCoverage() async {
  final testDir = Directory('test');
  final context = path.Context(style: path.Style.posix);
  final testFiles = testDir
      .listSync(recursive: true, followLinks: false)
      .where((e) =>
          FileSystemEntity.isFileSync(e.path) && e.path.endsWith('_test.dart'))
      .map(
          (tf) => context.normalize(path.relative(tf.path, from: testDir.path)))
      .toList()
        ..sort();
  final content = StringBuffer('''
// ignore_for_file: await_only_futures

import 'dart:async';

''');
  final executions = StringBuffer();
  for (var i = 0; i < testFiles.length; i++) {
    final testFile = testFiles[i];
    content.writeln("import '$testFile' as i$i;");
    executions.writeln('  i$i.main();');
  }
  content
    ..writeln('Future<void> main() async {')
    ..writeln()
    ..writeln(executions)
    ..writeln('}');
  File('test/all.dart').writeAsStringSync(content.toString());
  PubApp.global('dart_style')
      .run(['-w', '--fix']..add('test/all.dart'), script: 'format');
}

Answer 2

The 'Run with Coverage' option in Android Studio is enabled only for Flutter integration tests for some reason (at least for me).

I wrote an article that describes how to generate code coverage reports locally and on CodeCov and CoverAlls that should get you close to what you want to do. Includes all source code and shows live examples.

It works for both flutter and dart packages.

You can find it here: https://medium.com/@nocnoc/combined-code-coverage-for-flutter-and-dart-237b9563ecf8