Not using IsolationActivity via AndroidBenchmarkRunner

Question

I'm trying to try out Android's Jetpack Benchmark library and cannot seem to get past this error. Have tried everything I can think of, would appreciate any help making this simple example work.

java.lang.AssertionError: ERRORS (not suppressed): ACTIVITY-MISSING
(Suppressed errors: DEBUGGABLE EMULATOR UNLOCKED)

WARNING: Not using IsolationActivity via AndroidBenchmarkRunner
AndroidBenchmarkRunner should be used to isolate benchmarks from interference
from other visible apps. To fix this, add the following to your module-level
build.gradle:
android.defaultConfig.testInstrumentationRunner
= "androidx.benchmark.junit4.AndroidBenchmarkRunner"

I have followed the sample guide here, and my defaultConfig contains:

testInstrumentationRunner 'androidx.benchmark.junit4.AndroidBenchmarkRunner'
// Suppressing errors I can ignore while I'm testing. Unfortunately, suppressing 
// the ACTIVITY-MISSING error causes runtime crashes
testInstrumentationRunnerArgument 'androidx.benchmark.suppressErrors', 'EMULATOR,DEBUGGABLE,UNLOCKED'

and my dependencies contain:

    androidTestImplementation 'androidx.annotation:annotation:1.1.0'
    androidTestImplementation 'androidx.test:runner:1.2.0'
    androidTestImplementation 'androidx.test:core:1.2.0'
    androidTestImplementation 'androidx.test:rules:1.2.0'
    androidTestImplementation 'org.hamcrest:hamcrest-library:1.3'
    androidTestImplementation 'androidx.test.espresso:espresso-core:3.2.0'
    androidTestImplementation 'androidx.test.ext:junit:1.1.1'
    androidTestImplementation 'junit:junit:4.12'
    androidTestImplementation 'androidx.benchmark:benchmark-junit4:1.0.0'

My benchmark code is as follows:

import android.util.Log;

import androidx.benchmark.BenchmarkState;
import androidx.benchmark.junit4.BenchmarkRule;
import androidx.test.ext.junit.runners.AndroidJUnit4;

import org.junit.Rule;
import org.junit.Test;
import org.junit.runner.RunWith;

@RunWith(AndroidJUnit4.class)
public class BenchmarkTest {

    @Rule
    public BenchmarkRule benchmarkRule = new BenchmarkRule();

    @Test
    public void log() {
        final BenchmarkState state = benchmarkRule.getState();
        while (state.keepRunning()) {
            Log.d("LogBenchmark", "the cost of writing this log method will be measured");
        }
    }
}

Answer 1

I faced the same issue when I followed the quick start guide. However, I got it to work by adding a new benchmark module and picking up the dependencies from the sample app.

Steps to add the benchmark module:

1. File > New > New Module

2. Select Benchmark Module

Dependencies:

    implementation fileTree(dir: 'libs', include: ['*.jar'])

    androidTestImplementation project("<enter your project>")
    androidTestImplementation "org.jetbrains.kotlin:kotlin-stdlib:$kotlin_version"

    androidTestImplementation 'junit:junit:4.12'

    androidTestImplementation 'androidx.test:runner:1.2.0'
    androidTestImplementation 'androidx.test:rules:1.2.0'
    androidTestImplementation 'androidx.test.ext:junit:1.1.1'

    androidTestImplementation 'androidx.appcompat:appcompat:1.1.0'
    androidTestImplementation 'androidx.constraintlayout:constraintlayout:1.1.3'
    androidTestImplementation 'androidx.cardview:cardview:1.0.0'
    androidTestImplementation 'androidx.recyclerview:recyclerview:1.0.0'
    androidTestImplementation 'androidx.benchmark:benchmark-junit4:1.1.0-alpha01'

 buildTypes {
        debug {
            minifyEnabled true
            proguardFiles getDefaultProguardFile("proguard-android-optimize.txt"), "benchmark-proguard-rules.pro"
        }
        release {
            isDefault = true
            isDefault = true
            minifyEnabled true
            proguardFiles getDefaultProguardFile('proguard-android-optimize.txt'), 'benchmark-proguard-rules.pro'
            signingConfig signingConfigs.debug
        }
    }

Be sure to define the release signing config, since creating a benchmark module does not add it by default.