Visualize taps/gestures for flutter-test (or flutter-driver)

Question

When using flutter_driver / flutter_test, we simulate user behavior by doing things like await tap(). However, I want to see where is tapped on the screen of emulator. Is it possible? Thanks!

Answer 1

My idea for this (since Flutter Driver and widget tests do not use real taps) is to record the taps on a Flutter-level, i.e. using Flutter hit testing.

I will present you with a widget that you can wrap your app in to visualize and capture all taps. I wrote a complete widget for this.

Demonstration

Here is the result when wrapping the widget around the default template demo app:

Implementation

What we want to do is simple: react to all tap events in the size of our widget (the whole app is our child).
However, it comes with a little challenge: GestureDetector e.g. will not let taps through after reacting to them. So if we were to use a TapGestureRecognizer, we could either not react to taps that hit buttons in our app or we would not be able to hit the buttons (would only see our indication).

Therefore, we need to use our own render object to do the job. This is not a hard task when you a familiar with - RenderProxyBox is just the abstraction we need :)

Catching hit events

By overriding hitTest, we can make sure that we always record hits:

@override
bool hitTest(BoxHitTestResult result, {required Offset position}) {
  if (!size.contains(position)) return false;
  // We always want to add a hit test entry for ourselves as we want to react
  // to each and every hit event.
  result.add(BoxHitTestEntry(this, position));
  return hitTestChildren(result, position: position);
}

Now, we can use handleEvent to both record the hit events and also visualize them:

@override
void handleEvent(PointerEvent event, covariant HitTestEntry entry) {
  // We do not want to interfere in the gesture arena, which is why we are not
  // using regular tap recognizers. Instead, we handle it ourselves and always
  // react to the hit events (ignoring the gesture arena).
  if (event is PointerDownEvent) {
    // Record the global position.
    recordTap(event.position);

    // Visualize local position.
    visualizeTap(event.localPosition);
  }
}

Visualizing

I will spare you the details (full code at the end): I decided to create an AnimationController for each recorded hit and store that along with the local position.

Since we are using a RenderProxyBox, we can just call markNeedsPaint when the animation controller fires and then paint a circle for all recorded taps:

@override
void paint(PaintingContext context, Offset offset) {
  context.paintChild(child!, offset);

  final canvas = context.canvas;
  for (final tap in _recordedTaps) {
    drawTap(canvas, tap);
  }
}

Code

Of course, I glanced over most parts of the implementation because you can just read through them :)
The code should be straight-forward since I outlined the concepts I used.

You can find the full source code here.

Usage

The usage is straight-forward:

TapRecorder(
  child: YourApp(),
)

Even in my example implementation, you can configure the tap circle color, size, duration, etc.:

/// These are the parameters for the visualization of the recorded taps.
const _tapRadius = 15.0,
    _tapDuration = Duration(milliseconds: 420),
    _tapColor = Colors.white,
    _shadowColor = Colors.black,
    _shadowElevation = 2.0;

You could make them widget parameters if you wanted to.

Testing

I hope that the visualization part lives up to your expectations.

If you want to go beyond that, I made sure that the taps are stored globally:

/// List of the taps recorded by [TapRecorder].
///
/// This is only a make-shift solution of course. This will only be viable
/// when using a single [TapRecorder] because it is saved as a top-level
/// variable.
@visibleForTesting
final recordedTaps = <Offset>[];

You can simply access the list in tests to check the recorded taps :)

The end

I had a lot of fun implementing this and I hope it met your expectations.
The implementation is just a quick make-shift one, however, I hope it can provide you with all the concepts needed to take this idea to a good level :)