Android Test Sharding

Question

Can anyone explain what is test sharding in android mean to accomplish? And if someone could share any tutorial would be really helpful.

The word shard means a small part of a whole. How does sharding is performed on the basis of just a number, and on what basis should I specify the shardIndex?

Definition as in developer docs.

Test sharding

The test runner supports splitting a single test suite into multiple shards, so you can easily run tests belonging to the same shard together as a group, under the same Instrumentation instance. Each shard is identified by an index number. When running tests, use the -e numShards option to specify the number of separate shards to create and the -e shard index option to specify which shard to run.

For example, to split the test suite into 10 shards and run only the tests grouped in the second shard, use the following command:

adb shell am instrument -w -e numShards 10 -e shardIndex 2

Answer 1

Test sharding allows you to evenly divide up your tests into groups. The shard index is which "percentage" group you are running. How the groups are divided is arbitrary as the point of sharding is to parallelize your tests.

For example, lets say you have 60 tests to run and each tests takes 1 minute to complete. If you were to run this on a single device it would take one hour to run all tests. Now lets say you'd like to speed up your tests by running half the tests on one device and the other half on another device at the same time thus taking only 30 minutes in total.

You can do this by running the following ADB commands in parallel.

adb -s DEVICE_1_SERIAL shell am instrument -w -e numShards 2 -e shardIndex 0 > device_1_results // Runs first half of the tests
adb -s DEVICE_2_SERIAL shell am instrument -w -e numShards 2 -e shardIndex 1 > device_2_results // Runs second half of the tests

You now have run all 60 tests in only 30 minutes by spreading the load evenly to two devices and can now process the results.

For the nitty gritty on how it works, look at the ShardingFilter inside of https://android.googlesource.com/platform/frameworks/testing/+/2fe8aed7542ee05ce504d69656475d1948e9c5b2/androidtestlib/src/com/android/test/runner/TestRequestBuilder.java

Answer 2

You can also achieve this via gradle commands. If you gradle tasks to run your unit or ui tests below command will allow you to filter tests based on the shard number and index.

ANDROID_SERIAL=emulator-5554 ./gradlew connectedAndroidTest -Pandroid.testInstrumentationRunnerArguments.numShards=3 -Pandroid.testInstrumentationRunnerArguments.shardIndex=0

ANDROID_SERIAL value can be changed to device id pulled from adb devices