Conflicting library version in a java maven project

Question

When building a Maven project that has many dependencies, some of those dependencies depend on the same library but use a different version which is causing errors when running an application.

For example, if I add two different project dependencies, A and B that both depend on Apache Commons HTTP client but each one on a different version, once the class-loader loads A's Apache commons http client classes, B will try to use them since they are already loaded by the class loader.

But B's bytecode depends on a different version of the loaded classes causing multiple problems when running the application. A common one is method-not-found exception (since A's version of http client doesn't use a specific method any more).

What is the general strategy when building to avoid such conflicts? Does one have to manually check the dependency tree to figure out which common libraries collide with each other?

Answer 1

You can use the tree goal of the Maven dependency plugin to display all transitive dependencies in your project and look for dependencies that say "omitted for conflict".¹

mvn dependency:tree -Dverbose mvn dependency:tree -Dverbose | grep 'omitted for conflict' 

Once you know which dependency has version conflicts, you can use the includes parameter to show just dependencies that lead to that one to see how a particular dependency is being pulled in. For example, a project where different versions of C are pulled in by A and B:

mvn dependency:tree -Dverbose -Dincludes=project-c  [INFO] com.my-company:my-project:jar:1.0-SNAPSHOT [INFO] +- project-a:project-a:jar:0.1:compile [INFO] |  \- project-c:project-c:jar:1.0:compile [INFO] \- project-b:project-b:jar:0.2:compile [INFO]    \- project-x:project-x:jar:0.1:compile [INFO]       \- (project-c:project-c:jar:2.0:compile - omitted for conflict) 

To actually resolve the conflict, in some cases it may be possible to find a version of the transitive dependency that both of your primary dependencies will work with. Add the transitive dependency to the dependencyManagement section of your pom and try changing the version until one works.

However, in other cases it may not be possible to find a version of the dependency that works for everyone. In these cases, you may have to step back the version on one of the primary dependencies in order to make it use a version of the transitive dependency that works for everybody. For instance, in the example above, A 0.1 uses C 1.0 and B 0.2 uses C 2.0. Assume C 1.0 and 2.0 are completely incompatible. But maybe it is possible for your project to use B 0.1 instead, which happens to depend on C 1.5, which is compatible with C 1.0.

Of course these two strategies will not always work, but I have found success with them before. Other more drastic options include packaging your own version of the dependency that fixes the incompatibility or trying to isolate the two dependencies in separate classloaders.

Answer 2

Welcome to maven dependency hell, as it's fondly known. This is a somewhat common problem as projects grow and more external dependencies are introduced.

Besides Apache Commons (mentioned in your original question), logging frameworks (log4j, slf4j) are another frequent culprit.

I agree with the advice given by "matts" on how to resolve conflicts once they are identified. In terms of catching these version conflicts early, you can also use the maven "enforcer" plugin. Refer to the "dependencyConvergence" config. Also see this SO post.

Using the enforcer plugin will fail the build immediately on version conflict, which saves you from the manual checks. This is an aggressive strategy, but prevents the type of run-time problems that prompted your question/post. Like anything, the enforcer plugin has pros and cons. We started using it within the last year, but then discovered it can be a blessing and a curse. Many versions of libs/frameworks are backwards compatible, and so depending (whether directly or indirectly) on both version 1.2.3 and 1.2.4 is often fine at both compile-time and run-time. However, the enforcer plugin will flag this conflict and require you to declare exactly which version you want. Assuming the number of dependency-conflicts is small, this does not require much work. However, once you introduce a large framework (e.g. Spring MVC) it can get nasty.

Hopefully that's useful information.