Best practice for dependency management in a project with a huge number of dependencies

Question

Our project is like an adapter/facade interface for a huge amount of different other libraries. The dependencies are somehow overlapped, sometimes in conflict or sometimes even make project breaks in silence for wrong version of dependencies provide wrong behavior of same interface. We are using Ivy and Ant to do basic dependencies management. What's the best practice to manage dependencies and detect wrong behavior early on?

Answer 1

The important part of this question is about process, not tools.

If a project's dependencies are owned by other teams or third parties, that project must explicitly accept each new version of each new dependency. Allowing dependencies to upgrade themselves would allow them to break the depending project without warning, which is what it sounds like is happening.

Each dependency must release known versions, whether as binaries or tags in version control or whatever is appropriate to your stack. Each time a project wants to upgrade a dependency, it must test the result of the upgrade. (As usual comprehensive automated testing will be a big help.) If it fails (either because the dependency is just broken or because the dependency brings in an incompatible version of a transitive dependency), abandon the upgrade, report the problem to the owners of the dependencies, and try again after they've released a version which fixes the problem. If it succeeds, change the version of the dependency that the project uses in its build configuration.

Ideally a project will upgrade dependencies one at a time and fully test each upgrade separately. If it's necessary to upgrade more than one dependency all at once (perhaps because two dependencies both depend on a third dependency of which there can only be one version in the system) that's fine, although it's a bigger change and thus riskier. If your project has transitive dependencies like this, it will be worth the engineering effort to make them backward-compatible over as many versions as is reasonable.

Of course many tools support this process easily enough: just pin each dependency to a specific version.