What should I do about dependency conflicts when using the maven-shade-plugin?

Question

I'm using the maven-shade-plugin to create an executable jar that contains all of my project's dependencies. Sometimes, these dependencies bring in dependencies of their own that clash with the dependencies of other libraries, and the maven-shade-plugin warns me that it isn't sure which version to include in the uber jar.

[WARNING] maven-shade-plugin has detected that some .class files
[WARNING] are present in two or more JARs. When this happens, only
[WARNING] one single version of the class is copied in the uberjar.
[WARNING] Usually this is not harmful and you can skeep these
[WARNING] warnings, otherwise try to manually exclude artifacts
[WARNING] based on mvn dependency:tree -Ddetail=true and the above
[WARNING] output

In general, my response to this warning is to use the <exclusions> element of the dependency declaration in my pom file to remove the offending dependencies from my project:

<!-- Amazon ElastiCache Client -->
<dependency>
    <groupId>com.amazonaws</groupId>
    <artifactId>elasticache-java-cluster-client</artifactId>
    <version>1.0.61.0</version>
    <exclusions>
        <!-- this junit dependency clashes with our test-scoped one and causes integration tests to fail to run -->
        <exclusion>
            <groupId>junit</groupId>
            <artifactId>junit-dep</artifactId>
        </exclusion>
        <!-- this dependency brings in two versions of cglib that clash with one another -->
        <exclusion>
            <groupId>jmock</groupId>
            <artifactId>jmock-cglib</artifactId>
        </exclusion>
        <!-- newer versions of these dependencies come with dropwizard-core -->
        <exclusion>
            <groupId>log4j</groupId>
            <artifactId>log4j</artifactId>
        </exclusion>
        <exclusion>
            <groupId>commons-logging</groupId>
            <artifactId>commons-logging</artifactId>
        </exclusion>
      </exclusions>
</dependency>

When I do this, I use mvn dependency:tree to make sure that I'm excluding the lower version of the offending dependency, in hopes that the newest version is the most mature and bug free.

Cases like the one above that end up with a lot of exclusions raise two questions about this practice:

1. In the example above, why do I have to manually exclude junit and jmock? Both of these dependencies are marked as <scope>test</scope> in the elasticache-java-cluster-client pom.xml, so I would expect that they wouldn't be included in the jar that I get from maven.
2. While my practice of always taking the newer version of a dependency seems to have worked so far, I'm afraid that one of these days I'm going to break something. Is there a better way to determine which version of an dependency to keep?

Answer 1

Have you tried adding the maven-enforcer-plugin with the DependencyConvergence rule? This worked well for me in combination with the shade plugin. It will tell you which artifacts are bringing in different versions of the same classes. It allowed me to find out what I have to exclude.

        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-enforcer-plugin</artifactId>
            <executions>
                <execution>
                    <id>enforce</id>
                    <configuration>
                        <rules>
                            <DependencyConvergence/>
                        </rules>
                    </configuration>
                    <goals>
                        <goal>enforce</goal>
                    </goals>
                </execution>
            </executions>
        </plugin>