How to speed up my gradle build and skip failing dependency lookups?

Question

I have a an android gradle build that looks for dependencies on four different servers.

Now the build takes up to two minutes because those dependencies are checked every build. Maybe this occurs because one of the repositories returns an error for most of the dependencies.

I have the following sources in my gradle file:

repositories {
   maven { url 'http://jenkins.local:8081/nexus/content/groups/public/' }
   mavenCentral()
   maven { url 'https://oss.sonatype.org/content/repositories/snapshots/'}
   maven {url "https://repo.commonsware.com.s3.amazonaws.com"}
}

The first entry is a proxy that should cache all artifacts after the first request. This is a part of the dependency list:

compile 'com.android.support:support-v4:19.+'

compile 'com.squareup.picasso:picasso:2.2.+'
compile 'com.commonsware.cwac:merge:1.0.1'
compile files('libs/crittercism_v4_3_0_sdkonly.jar')
testCompile 'junit:junit:4.10'
testCompile 'org.robolectric:robolectric:2.3-SNAPSHOT'
testCompile 'com.squareup:fest-android:1.0.+'
testCompile 'org.mockito:mockito-all:1.9.5'
instrumentTestCompile 'junit:junit:4.10'
instrumentTestCompile 'org.robolectric:robolectric:2.3-SNAPSHOT'
instrumentTestCompile 'com.squareup:fest-android:1.0.+'
instrumentTestCompile 'org.mockito:mockito-all:1.9.5'

Now while running gradle I get the following output (Again a snapshot):

Failed to get resource: GET. [HTTP HTTP/1.1 403 Forbidden: https://repo.commonsware.com.s3.amazonaws.com/de/greenrobot/greendao/]
Failed to get resource: GET. [HTTP HTTP/1.1 403 Forbidden: https://repo.commonsware.com.s3.amazonaws.com/com/android/support/support-v4/maven-metadata.xml]
Failed to get resource: GET. [HTTP HTTP/1.1 403 Forbidden: https://repo.commonsware.com.s3.amazonaws.com/com/android/support/support-v4/]
Failed to get resource: GET. [HTTP HTTP/1.1 403 Forbidden: https://repo.commonsware.com.s3.amazonaws.com/com/squareup/picasso/picasso/maven-metadata.xml]
Failed to get resource: GET. [HTTP HTTP/1.1 403 Forbidden: https://repo.commonsware.com.s3.amazonaws.com/com/squareup/picasso/picasso/]
Failed to get resource: GET. [HTTP HTTP/1.1 403 Forbidden: https://repo.commonsware.com.s3.amazonaws.com/com/squareup/picasso/picasso/maven-metadata.xml]
Failed to get resource: GET. [HTTP HTTP/1.1 403 Forbidden: https://repo.commonsware.com.s3.amazonaws.com/com/squareup/picasso/picasso/]
Failed to get resource: GET. [HTTP HTTP/1.1 403 Forbidden: https://repo.commonsware.com.s3.amazonaws.com/com/android/support/appcompat-v7/maven-metadata.xml]
Failed to get resource: GET. [HTTP HTTP/1.1 403 Forbidden: https://repo.commonsware.com.s3.amazonaws.com/com/android/support/appcompat-v7/]
Failed to get resource: GET. [HTTP HTTP/1.1 403 Forbidden: https://repo.commonsware.com.s3.amazonaws.com/com/android/support/mediarouter-v7/maven-metadata.xml]
Failed to get resource: GET. [HTTP HTTP/1.1 403 Forbidden: https://repo.commonsware.com.s3.amazonaws.com/com/android/support/mediarouter-v7/]
Failed to get resource: GET. [HTTP HTTP/1.1 403 Forbidden: https://repo.commonsware.com.s3.amazonaws.com/com/android/support/support-v4/maven-metadata.xml]
Failed to get resource: GET. [HTTP HTTP/1.1 403 Forbidden: https://repo.commonsware.com.s3.amazonaws.com/com/android/support/support-v4/]
Failed to get resource: GET. [HTTP HTTP/1.1 403 Forbidden: https://repo.commonsware.com.s3.amazonaws.com/com/fasterxml/jackson/core/jackson-databind/maven-metadata.xml]
Failed to get resource: GET. [HTTP HTTP/1.1 403 Forbidden: https://repo.commonsware.com.s3.amazonaws.com/com/fasterxml/jackson/core/jackson-databind/]
Failed to get resource: GET. [HTTP HTTP/1.1 403 Forbidden: https://repo.commonsware.com.s3.amazonaws.com/com/fasterxml/jackson/core/jackson-core/maven-metadata.xml]
Failed to get resource: GET. [HTTP HTTP/1.1 403 Forbidden: https://repo.commonsware.com.s3.amazonaws.com/com/fasterxml/jackson/core/jackson-core/]
Failed to get resource: GET. [HTTP HTTP/1.1 403 Forbidden: https://repo.commonsware.com.s3.amazonaws.com/com/fasterxml/jackson/core/jackson-annotations/maven-metadata.xml]
Failed to get resource: GET. [HTTP HTTP/1.1 403 Forbidden: https://repo.commonsware.com.s3.amazonaws.com/com/fasterxml/jackson/core/jackson-annotations/]
Failed to get resource: GET. [HTTP HTTP/1.1 403 Forbidden: https://repo.commonsware.com.s3.amazonaws.com/com/squareup/okhttp/okhttp/maven-metadata.xml]
Failed to get resource: GET. [HTTP HTTP/1.1 403 Forbidden: https://repo.commonsware.com.s3.amazonaws.com/com/squareup/okhttp/okhttp/]
Failed to get resource: GET. [HTTP HTTP/1.1 403 Forbidden: https://repo.commonsware.com.s3.amazonaws.com/com/squareup/retrofit/retrofit/maven-metadata.xml]
Failed to get resource: GET. [HTTP HTTP/1.1 403 Forbidden: https://repo.commonsware.com.s3.amazonaws.com/com/squareup/retrofit/retrofit/]
Failed to get resource: GET. [HTTP HTTP/1.1 403 Forbidden: https://repo.commonsware.com.s3.amazonaws.com/de/greenrobot/greendao/maven-metadata.xml]
Failed to get resource: GET. [HTTP HTTP/1.1 403 Forbidden: https://repo.commonsware.com.s3.amazonaws.com/de/greenrobot/greendao/]
Failed to get resource: GET. [HTTP HTTP/1.1 403 Forbidden: https://repo.commonsware.com.s3.amazonaws.com/com/android/support/support-v4/maven-metadata.xml]
Failed to get resource: GET. [HTTP HTTP/1.1 403 Forbidden: https://repo.commonsware.com.s3.amazonaws.com/com/android/support/support-v4/]
Failed to get resource: GET. [HTTP HTTP/1.1 403 Forbidden: https://repo.commonsware.com.s3.amazonaws.com/com/squareup/picasso/picasso/maven-metadata.xml]
Failed to get resource: GET. [HTTP HTTP/1.1 403 Forbidden: https://repo.commonsware.com.s3.amazonaws.com/com/squareup/picasso/picasso/]
Failed to get resource: GET. [HTTP HTTP/1.1 403 Forbidden: https://repo.commonsware.com.s3.amazonaws.com/com/squareup/picasso/picasso/maven-metadata.xml]
Failed to get resource: GET. [HTTP HTTP/1.1 403 Forbidden: https://repo.commonsware.com.s3.amazonaws.com/com/squareup/picasso/picasso/]
Failed to get resource: GET. [HTTP HTTP/1.1 403 Forbidden: https://repo.commonsware.com.s3.amazonaws.com/com/android/support/appcompat-v7/maven-metadata.xml]
Failed to get resource: GET. [HTTP HTTP/1.1 403 Forbidden: https://repo.commonsware.com.s3.amazonaws.com/com/android/support/appcompat-v7/]
Failed to get resource: GET. [HTTP HTTP/1.1 403 Forbidden: https://repo.commonsware.com.s3.amazonaws.com/com/android/support/mediarouter-v7/maven-metadata.xml]
Failed to get resource: GET. [HTTP HTTP/1.1 403 Forbidden: https://repo.commonsware.com.s3.amazonaws.com/com/android/support/mediarouter-v7/]
Failed to get resource: GET. [HTTP HTTP/1.1 403 Forbidden: https://repo.commonsware.com.s3.amazonaws.com/com/android/support/support-v4/maven-metadata.xml]

How can I force the build to do this lookup only once a day or to skip lookups that failed last time or to take the first artifact that was found and skip all the other repos for artifacts that are already available?

Answer 1

The moral of this story: make sure that your Maven repository returns 404 for anything that is not actually there.

In the specific case of hosting a repository up on Amazon S3, grant "Everyone" the "List" permission using the Amazon S3 Manager, or otherwise follow the instructions from this StackOverflow answer.