Common libraries in a large team

Question

Assume you have five products, and all of them use one or more of the company's internal libraries, written by individual developers.

It sounds simple but in practice, I found it to be very difficult to maintain.

How do you deal with the following scenarios:

1. A developer unintentionally introduces a bug and breaks everything in production.

2. Every library has to mature, That means the API needs to evolve, so how do you deploy the updated version to production if every developer needs to update/test their code while they are extremely busy on other projects? Is this a resource and time issue?

3. Version control, deployment,and usage. Would you store this in one global location or force each project to use, say, svn:externals to "tie" a library?

I've found that it is extremely hard to come up with a good strategy. My own pet theory is this:

1. Each common library has to have a super-thorough set of tests or else it should never be common, even if it means someone else duplicates the effort. Duplicate untested code is better than common untested code (you break only one project).

2. Each common library has to have a dedicated maintainer (can be offset by a really good test suite in a smaller team).

3. Each project should check out the version of the library that is known to work with it. This means a developer does not have to get pulled away to update API usage, as the common code gets updated. Which it will be. Every non-trivial piece of code evolves over months and years.

Thank you for your thoughts on this!

Answer 1

You have a competing set of goals here. First, a library of reusable components must be open enough that people from the other projects can easily add to it (or submit components to it). If it's too difficult for them to do that, they'll build their own libraries, and ignore the common one, leading to a lot of duplicate code and wasted effort. On the other hand, you want to control the development of the library enough that you can ensure its quality.

I've been in this position. There's no easy answer. However, there are some heuristics that can help.

• Treat the library as an internal project. Release it on regular intervals. Ensure that it has a well-defined release procedure, complete with unit tests and quality assurance. And, most important, release often, so that new submissions to the library show up in the product frequently.
• Provide incentives for people to contribute to the library, rather than just making their own internal libraries.
• Make it easy for people to contribute to the library, and make the criteria clear-cut and well-defined (e.g., new classes must come with unit tests and documentation).
• Put one or two developers in charge of the library, and (IMPORTANT!) allocate time for them to work on it. A library that is treated as an afterthought will quickly become an afterthought.

In short, model the development and maintenance of your internal library after a successful open source library project.

Answer 2

I don't agree with this:

Duplicate untested code is better than common untested code (you break only one project).

If you are all equally likely to create bugs by implementing the same thing, then you'll all have to fix potentially different bugs in each instance of the "duplicate" library.

It also seems that it'd be much faster/cheaper to write the library once and, instead of having multiple other teams write the same thing, have some resources allocated to testing.

---

Now to solve your actual problem: I'd mimic what we do with real third-party libraries. We use a particular version until we're ready, or compelled to upgrade. I don't upgrade everything just because I can--there has to be a reason.

Once I see that reason (bug fix, new feature, etc.), then I upgrade with the risk that the new library may have new bugs or breaking changes.

So, you're library project would continue development as necessary, without impacting individual teams until they were ready to "upgrade".

You could publish releases or peg/branches/tag svn to help with all this.

If all teams have access to the bug tracker, they could easily see what known issues exist in the upgrade-candidate before they upgrade, too. Or, you could maintain that list yourself.

@Brian Clapper provides some excellent guidelines for how to run your library as a project in his answer.