Dependency management for big projects

Question

I am working in a fairly big project. We have lots of projects and each project has dependencies. We use maven and normally we don't have any problems. So, without giving much details, imagine that for a given project, say, tps-reports the dependencies section of the pom.xml looks like:

  <name>TPS-Reports</name>
  <description>
   TPS Reports frontend.
  </description>
  <dependencies>
   <dependency>
    <groupId>com.initech</groupId>
    <artifactId>gui-components</artifactId>
    <version>2.5</version>
   </dependency>
   <dependency>
    <groupId>com.initech</groupId>
    <artifactId>multithreading</artifactId>
    <version>3.7</version>
   </dependency>
   <dependency>
    <groupId>com.initech</groupId>
    <artifactId>utils</artifactId>
    <version>2.3.0.0</version>
   </dependency>
   <dependency>
    <!-- TODO: update to new version -->
    <groupId>com.initech</groupId>
    <artifactId>object-pooling</artifactId>
    <version>1.9.3.1</version>
   </dependency>
  </dependencies>

Now, Initech has tons of projects that depend on, say, object-pooling, which also depends on many other more components, such as (utils and multithreading).

Life is good for object-pooling developers. It is a pretty stable module and it all goes well. As any other module, it also has dependencies. object-pooling developers are all gentlemen and fair ladies and whenever they find a critical bug, they update all projects that depend on object-pooling.

Now, version 1.9.3.1 of object-pooling is stable and has no known critical bugs. Developers work really hard to add a ton of new features and after some time, they release version 2.0.0.0. Of course, between 1.9.3.1 and 2.0.0.0, there are intermediate releases (e.g. 1.9.3.1, 1.9.3.2, 1.9.4.0, 1.9.5.3 and so on). As I said, life is good for object-pooling developers. Version 2.0.0.0 has new features and lots of fixes.

However, hell is just around the corner for tps-reports developers. They've been using 1.9.3.1 for quite a while now and, since there are no known bugs in this version, they're comfortable with an old version. Now, they want to use the revamped object-pooling, so they update their pom.xml to use version 2.0.0.0 and it now looks like this:

  <name>TPS-Reports</name>
  <description>
   TPS Reports frontend.
  </description>
  <dependencies>
   <dependency>
    <groupId>com.initech</groupId>
    <artifactId>gui-components</artifactId>
    <version>2.5</version>
   </dependency>
   <dependency>
    <groupId>com.initech</groupId>
    <artifactId>multithreading</artifactId>
    <version>3.7</version>
   </dependency>
   <dependency>
    <groupId>com.initech</groupId>
    <artifactId>utils</artifactId>
    <version>2.3.0.0</version>
   </dependency>
   <dependency>
    <!-- use object poooling's new features -->
    <groupId>com.initech</groupId>
    <artifactId>object-pooling</artifactId>
    <version>2.0.0.0</version>
   </dependency>
  </dependencies>

They discover that they have new bugs. Needless to say, these bugs did not exist when they depended on version 1.9.3.1 of object-pooling. They dig into their code repository's log and they discover that, not only the object-pooling guys have done thousands of commits, but also that they are using the newest versions of multithreading and utils, which also have lot of commits.

There are, obviously, a myriad of places where the problem can reside. It can be on object-pooling, it can be in the interaction between object-pooling and tps-reports, it can be on multithreading or utils or any weird combination.

The question(s) is(are): How do you guys go around this kind of problems? How do you manage dependencies on big projects that in turn depend on other projects? Are there some tools out there to assist on this task?

Thanks!

Answer 1

Sorry, no silver bullet here: unit testing is the answer. The bigger the project gets, the more important automatic testing becomes.

In your case, even if the bugs arise in manual testing it eventually comes down to particular case of using the library and you may be able to reduce this to a unit test. The test will pass in 1.9.3.1 and fail in 2.0.0.0.

Now you can send the test case to the object-pooling developers and tell them you are not upgrading until they make this and other tests pass. This will make their life a bit like yours and given enough test cases, your life will eventually be more like theirs :-)

If the bug is in their dependent library, they will have to do the same thing to their downstream developers.