How to integrate firebase test lab with circleci

Question

Hi I want to test my app on firebase test lab. I want to integrate it with circleci. I have read this document https://circleci.com/docs/1.0/firebase-test-lab/ and created config.yml

I have created GCLOUD_SERVICE_KEY in the environment variable of circleci but it is not testing app on firebase test lab.

config.yml

version: 2.0

defaults: &defaults
    docker:
      - image: circleci/android:api-27-alpha
    working_directory: ~/github-jobs
    environment:
      JAVA_TOOL_OPTIONS: "-Xmx1024m"
      GRADLE_OPTS: "-Dorg.gradle.daemon=false -Dorg.gradle.workers.max=2 -Dkotlin.incremental=false"
      TERM: dumb

gcloud_config: &gcloud_config
  working_directory: ~/github-jobs
  docker:
    - image: google/cloud-sdk:latest
  environment:
    TERM: dumb

update_sdk: &update_sdk
  name: Update SDK
  command: |
    mkdir "$ANDROID_HOME/licenses" || true
    echo "d56f5187479451eabf01fb78af6dfcb131a6481e" > "$ANDROID_HOME/licenses/android-sdk-license"
    echo "84831b9409646a918e30573bab4c9c91346d8abd" > "$ANDROID_HOME/licenses/android-sdk-preview-license"
    sdkmanager "platform-tools" "platforms;android-27"

jobs:
  build:
    <<: *defaults
    steps:
      - run:
          <<: *update_sdk
      - checkout
      - restore_cache:
          key: jars-{{ checksum "build.gradle" }}-{{ checksum  "app/build.gradle" }}-{{ checksum  "base/build.gradle" }}-{{ checksum  "instantapp/build.gradle" }}-{{ checksum  "main/build.gradle" }}-{{ checksum  "tv/build.gradle" }}
      - run:
          name: chmod permissions
          command: chmod +x ./gradlew
      - run:
          name: Download Dependencies
          command: ./gradlew dependencies --no-daemon
      - save_cache:
          paths:
            - ~/.gradle
          key: jars-{{ checksum "build.gradle" }}-{{ checksum  "app/build.gradle" }}-{{ checksum  "base/build.gradle" }}-{{ checksum  "instantapp/build.gradle" }}-{{ checksum  "main/build.gradle" }}-{{ checksum  "tv/build.gradle" }}
      - run:
          name: Assemble APKs
          command: ./gradlew assemble --no-daemon
      - save_cache:
          paths:
            - ~/.gradle/caches
            - ~/.gradle/wrapper
          key: jars-{{ checksum "build.gradle" }}-{{ checksum  "app/build.gradle" }}
      - store_artifacts:
          path: app/build/outputs/apk
          destination: apks
      - store_artifacts:
          path: app/build/reports
          destination: reports
      - store_artifacts:
          path: build/dependencyUpdates
          destination: dependencyUpdates
      - store_test_results:
          path: app/build/test-results
      - persist_to_workspace:
          root: .
          paths:
            - build
            - app/build

# Google Cloud Service

export_gcloud_key: &export_gcloud_key
  run:
    name: Export Google Cloud Service key environment variable
    command: echo 'export GCLOUD_SERVICE_KEY="$GCLOUD_SERVICE_KEY"' >> $BASH_ENV
decode_gcloud_key: &decode_gcloud_key
  run:
    name: Decode Google Cloud credentials
    command: echo $GCLOUD_SERVICE_KEY | base64 -di > ${HOME}/client-secret.json

test_instrumented:
  <<: *gcloud_config
  steps:
    - *export_gcloud_key
    - *decode_gcloud_key
    - run:
        name: Set Google Cloud target project
        command: gcloud config set project dazzling-fire-5515
    - run:
        name: Authenticate with Google Cloud
        command: gcloud auth activate-service-account [email protected] --key-file ${HOME}/client-secret.json
    - run:
        name: Run instrumented test on Firebase Test Lab
        command: gcloud firebase test android run --type instrumentation --app app/build/outputs/apk/debug/app-debug.apk --test app/build/outputs/apk/androidTest/debug/app-debug-androidTest.apk --device model=Nexus5X,version=26,locale=en_US,orientation=portrait --environment-variables coverage=true,coverageFile=/sdcard/tmp/code-coverage/connected/coverage.ec --directories-to-pull=/sdcard/tmp --timeout 20m
    - run:
        name: Create directory to store test results
        command: mkdir firebase
    - run:
        name: Download instrumented test results from Firebase Test Lab
        command: gsutil -m cp -r -U "`gsutil ls gs://test-lab-3udbiqpdyp0d0-miwcp7d69v80m | tail -1`*" /root/workspace/firebase/
    - persist_to_workspace:
        root: .
        paths:
          - firebase
    - store_artifacts:
        path: firebase/
        destination: /firebase/

general:
  branches:
    only:
     - master # list of branches to build
     - develop

Does anyone know what I am missing ?

Answer 1

For CirlceCI 2.0 there are just Firebase & GCP Deployment examples; which are not really applicable here... the gcloud part is rather relevant, because the docs at Using Firebase Test Lab with Continuous Integration Systems refer to it:

gcloud firebase test android run --app <local_server_path>/<app_apk>.apk --test <local_server_path>/<app_test_apk>.apk

On CircleCI 2.0 the gcloud command appears to be sudo /opt/google-cloud-sdk/bin/gcloud. as well as it is being described here, a) how to obtain the exit codes and b) what they actually mean: https://firebase.google.com/docs/test-lab/android/command-line#running_your_instrumentation_tests

The problem description of "is not testing app" is not sufficient to tell what exactly is wrong. I'd assume, the command initiating the instrumentation test should throw some error or some complaint; please provide further details (logs).

I've just found one article on Medium, which seems to be what you're looking for: All you need to know about CircleCI 2.0 with Firebase Test Lab (there is another one article, which is also utilizing FastLane).