Gradle - download dependencies, lock versions and update dependencies manually

Question

The issue.

Gradle dependency management made so:

• there is no easy way to check availability of dependencies updates (only using some third-party plugins like ben-manes/gradle-versions-plugin) and download updates replacing old versions;
• dependencies artifacts are downloaded from remote repositories and then stored in gradle cache and reused in succeeding builds; but successful compilation of your project must not depend on having a connection to Internet, availability of remote repositories and existence of specific versions of dependencies in these repositories.

The goal.

• download and store all dependencies artifacts in VCS;
• manually check updates for these dependencies and download them.

Answer 1

My solution works for Gradle configuration using java or android plugins.

java plugin defines compile and testCompile configurations. compile is for dependencies that are required to compile the production source of the project. testCompile is for dependencies that are required to compile the test source of the project.

Let's define our own configurations in build.gradle:

configurations {
    download
    testDownload
}

Next let's create directories:

• libs/compile/downloaded is where download dependencies will be stored;
• libs/testCompile/downloaded is where testDownload dependencies will be stored.

Next we define several tasks.

Delete dependencies from download configuration:

task cleanDownloadedDependencies(type: Delete) {
    delete fileTree('libs/compile/downloaded')
}

Delete dependencies from testDownload configuration:

task cleanDownloadedTestDependencies(type: Delete) {
    delete fileTree('libs/testCompile/downloaded')
}

Download dependencies from download configuration:

task downloadDependencies(type: Copy) {
    from configurations.download
    into "libs/compile/downloaded/"
}

Download dependencies from testDownload configuration:

task downloadTestDependencies(type: Copy) {
    from configurations.testDownload
    into "libs/testCompile/downloaded/"
}

Execute all above tasks to update dependencies:

task updateDependencies {
    dependsOn cleanDownloadedDependencies, cleanDownloadedTestDependencies, downloadDependencies, downloadTestDependencies
}

Next we define our dependencies:

dependencies {
    download(
            'com.google.code.gson:gson:+',
            'joda-time:joda-time:+',
    )
    testDownload(
            'junit:junit:+'
    )

And then we tell where compile and testCompile configurations should take dependencies used for compilation.

    compile fileTree(dir: 'libs/compile', include: '**/*.jar')
    testCompile fileTree(dir: 'libs/testCompile', include: '**/*.jar')
}

Now you can download or update already downloaded dependencies:

./gradlew updateDependencies

If you are using android plugin then you can also add androidTestDownload configuration for dependencies that are required for compilation and running tests on Android device. Also some dependencies can be provided as aar artifacts.

This is the example for Gradle configuration using android plugin:

...

repositories {

    ...

    flatDir {
        dirs 'libs/compile', 'libs/compile/downloaded',
                'libs/testCompile', 'libs/testCompileDownloaded',
                'libs/androidTestCompile', 'libs/androidTestCompile/downloaded'
    }
}

configurations {
    download
    testDownload
    androidTestDownload
}

android {
    ...
}

dependencies {
    download(
            'com.android.support:support-v4:+',
            'com.android.support:appcompat-v7:+',
            'com.google.android.gms:play-services-location:+',
            'com.facebook.android:facebook-android-sdk:+',
            'com.vk:androidsdk:+',
            'com.crashlytics.sdk.android:crashlytics:+',
            'oauth.signpost:signpost-core:+',
            'oauth.signpost:signpost-commonshttp4:+',
            'org.twitter4j:twitter4j-core:+',
            'commons-io:commons-io:+',
            'com.google.code.gson:gson:+',
            'org.jdeferred:jdeferred-android-aar:+'
    )
    compile fileTree(dir: 'libs/compile', include: '**/*.jar')
    testCompile fileTree(dir: 'libs/testCompile', include: '**/*.jar')
    androidTestCompile fileTree(dir: 'libs/androidTestCompile', include: '**/*.jar')
}


task cleanDownloadedDependencies(type: Delete) {
    delete fileTree('libs/compile/downloaded')
}

task cleanDownloadedTestDependencies(type: Delete) {
    delete fileTree('libs/testCompile/downloaded')
}

task cleanDownloadedAndroidTestDependencies(type: Delete) {
    delete fileTree('libs/androidTestCompile/downloaded')
}

task downloadDependencies(type: Copy) {
    from configurations.download
    into 'libs/compile/downloaded/'
}

task downloadTestDependencies(type: Copy) {
    from configurations.testDownload
    into 'libs/testCompile/downloaded/'
}

task downloadAndroidTestDependencies(type: Copy) {
    from configurations.androidTestDownload
    into 'libs/androidTestCompile/downloaded/'
}

task updateDependencies {
    dependsOn cleanDownloadedDependencies, cleanDownloadedTestDependencies, cleanDownloadedAndroidTestDependencies, downloadDependencies, downloadTestDependencies, downloadAndroidTestDependencies
}

fileTree(dir: 'libs/compile', include: '**/*.aar')
        .each { File file ->
    dependencies.add("compile",
            [name: file.name.lastIndexOf('.').with { it != -1 ? file.name[0..<it] : file.name }, ext: 'aar'])
}

fileTree(dir: 'libs/testCompile', include: '**/*.aar')
        .each { File file ->
    dependencies.add("testCompile",
            [name: file.name.lastIndexOf('.').with { it != -1 ? file.name[0..<it] : file.name }, ext: 'aar'])
}

fileTree(dir: 'libs/androidTestCompile', include: '**/*.aar')
        .each { File file ->
    dependencies.add("androidTestCompile",
            [name: file.name.lastIndexOf('.').with { it != -1 ? file.name[0..<it] : file.name }, ext: 'aar'])
}

Answer 2

For locking versions of downloaded dependencies (library/ etc versions to hard-coded versions) for making the builds reproducible, now Gradle 4.8 and onwards, we'll have inbuilt "dependency lock" support. This will help greatly to make builds reproducible if someone is using dynamic versions (M.m.p/i) Major.minor.patch/interimBranch etc (ex: 4.+ or 3.1.+) or version range for pulling artifacts from a binary repository tool (ex: Artifactory / Nexus).

Any Gradle user who is using Gradle version 4.8+ should start using this new feature. https://docs.gradle.org/4.8/userguide/dependency_locking.html For Gradle 4.8 Release Notes: https://docs.gradle.org/4.8/release-notes.html

In past, this dependency lock feature was provided to Gradle community and made available via a FOSS plugin available to Gradle from Netflix Nebula's https://github.com/nebula-plugins/gradle-dependency-lock-plugin and https://plugins.gradle.org/plugin/nebula.dependency-lock