spring dependency-management gradle plugin doesn't download dependency

Question

I'm using spring dependency-management gradle plugin on IntelliJ. I've a root module with following

apply plugin: "io.spring.dependency-management"

dependencyManagement {
    dependencies {
      dependencySet(group: "org.apache.hadoop", version: "2.6.0-cdh5.14.4") {
                entry "hadoop-common"
                entry "hadoop-hdfs"
            }
     }
}

If I add

dependency 'org.apache.hadoop:hadoop-tools:2.6.0-mr1-cdh5.14.4' 

or

dependencySet(group: "org.apache.hadoop", version: "2.6.0-mr1-cdh5.14.4") {
    entry ("hadoop-tools") {
        exclude group: 'ch.qos.logback', module: 'logback-classic'
        exclude group: 'org.slf4j', module: 'slf4j-log4j12'
    }
}

in the root module, it doesn't download the hadoop-tools jar. It downloads this dependency only when I add the below in the sub module.

plugins {
    id "com.github.johnrengelman.shadow" version "2.0.4"
}

dependencies {
    compile ("org.apache.hadoop:hadoop-tools:2.6.0-mr1-cdh5.14.4") {
        exclude group: 'ch.qos.logback', module: 'logback-classic'
        exclude group: 'org.slf4j', module: 'slf4j-log4j12'
    }
}

Why is the behavior?

Answer 1

In order to understand this behavior you need to understand how Spring DependencyManagement plugin works (see this section in the official documentation ):

• the dependencyManagement { } block is used to configure the contraints that will apply to the dependencies (version to use, exclusions for transitive dependencies, etc..) but this block will not apply itself these dependencies to you project,
• the dependencies of your project must be configured using the dependencies{ } block

In your example:

• first you have configured the dependencyManagement block in root project with constraints on "hadoop-common" and "hadoop-hdfs" modules, then you added contraint on "hadoop-tools" (using "dependency" or "dependencySet" in the dependencyManagement block): at this stage, you have not explicitly added any dependencies to your projects, but only configured the dependencies contraints

  ==> this explains why "hadoop-tools" dependency is not added/downloaded to your project.

• then you added a "compile" dependency on "hadoop-tools" using the dependencies block , which is the correct way to declare dependencies, and this made the "hadoop-tools" lib available in your project.

If I understand well your requirement, based on the source code you provided in the question: you could configure your projects as follow:

root project's script

buildscript {
    repositories {
        jcenter()
    }
    dependencies {
        classpath "org.springframework.boot:spring-boot-gradle-plugin:2.0.4.RELEASE"
    }
}

// configure plugins to be applied and dependencies contraints for all subprojects
subprojects {
    apply plugin: 'java'
    apply plugin: io.spring.gradle.dependencymanagement.DependencyManagementPlugin

    dependencyManagement {
        dependencies {
            // set version for hadoop-common & hadoop-hdfs to "2.6.0-cdh5.14.4"
            dependencySet(group: "org.apache.hadoop", version: "2.6.0-cdh5.14.4") {
                entry "hadoop-common"
                entry "hadoop-hdfs"
            }
            // set version "2.6.0-mr1-cdh5.14.4" for hadoop-tool, 
            //   and exclude slf4j-log4j12 module from transitive dependencies
            dependency (group: "org.apache.hadoop" , name: "hadoop-tools", version : "2.6.0-mr1-cdh5.14.4") {
                exclude 'org.slf4j:slf4j-log4j12'
            }
        }
    }
    repositories {
        jcenter()
        maven {
            url = 'https://repository.cloudera.com/content/repositories/releases/'
        }
    }
}

sub-project script

dependencies{

    // hadoop-tools module version is defined (constrained) 
    //   by dependencyManagement in root project build script
    compile 'org.apache.hadoop:hadoop-tools'

}