Allow unstable Android Gradle builds on Jenkins

Question

Hi I have set up my Android project on Jenkins to provide CI. Its working well, running tests on a connected Android handset. The tests run on the Android Test Framework which extends jUnit3.

Unfortunately, the build is marked as a failure if there are any test failures. I'd like to be able to improve this in two ways:

1. Allowing unstable builds
2. Being able to mark known test failures

For item 1 I tried adding this to the project build.gradle:

connectedCheck {
    ignoreFailures = true
}

But it has no effect. Looking at the build log, I realised the actual test task is called connectedInstrumentTest, but this task is not found:

connectedInstrumentTest {
    ignoreFailures = true
}

causes:

Could not find method connectedInstrumentTest() for arguments [build_4ldpah0qgf0ukktofecsq41r98$_run_closure3@9cd826] on project ':Playtime'.

That am I missing?

Thanks

EDIT: Heres my project build.gradle, nothing special:

buildscript {
    repositories {
        mavenCentral()
    }
    dependencies {
        classpath 'com.android.tools.build:gradle:0.5.+'
    }
}
apply plugin: 'android'

dependencies {
    compile files('libs/android-support-v4.jar')
}

android {
    compileSdkVersion 17
    buildToolsVersion "18.0.1"

    defaultConfig {
        minSdkVersion 8
        targetSdkVersion 16
        testPackageName "com.bb.pt.test"
        testInstrumentationRunner "android.test.InstrumentationTestRunner"
    }
}

connectedCheck {
    ignoreFailures = true
}

My gradle settings in jenkins:

switches: --stacktrace --info
tasks: :pt:assembleDebug :pt:assembleTest :pt:connectedCheck

EDIT:

I built gradlew and tried that. Same output. I don't want the build to FAIL if there are test failures:

:pt:connectedInstrumentTest FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':pt:connectedInstrumentTest'.
> There were failing tests. See the report at: file:///home/simon/WorkingCopies/bb/code/trunk/pt/pt/build/reports/instrumentTests/connected/index.html

* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output.

BUILD FAILED

I tried to qualify the task name in build.gradle:

task connectedCheck {
    ignoreFailures = true
}

But it thinks I am trying to add a new task rather than modify the existing one.

FAILURE: Build failed with an exception.

* Where:
Build file '/home/simon/WorkingCopies/bb/code/trunk/pt/pt/build.gradle' line: 31

* What went wrong:
A problem occurred evaluating project ':pt'.
> Cannot add task ':pt:connectedCheck' as a task with that name already exists.

Answer 1

After our conversation I believe that:

• the problem is gradle configuration only and not jenkins related. Get it to work in gradle.

• in gradle I believe (though I am not an expert) you should get the connectedInstrumentTest to ignore failures, but your attempt to use the following failed

  connectedInstrumentTest {
      ignoreFailures = true
  }
  

• maybe the solution is to wrap this config node like this:

  project.gradle.taskGraph.whenReady {
    connectedInstrumentTest {
      ignoreFailures = true
    }
  }
  

https://github.com/stanfy/hotandroid/blob/master/part0/build.gradle