Can you develop directly in Git submodules?

Question

I have two projects (A & B). They both use project Common. I want to include Common into A & B via submodules because then I can directly tie each commit in A & B to which commit they rely on in Common.

In the past I tried to get my team using submodules like this, but we couldn't get it to work smoothly. We were developing Common code from within the submodule itself & committing from the submodule but we ran into so many problems that we reverted to having all projects under the same directory (C:\dev\A, C:\dev\Common).

I'm pretty sure we have no idea how submodules are supposed to be used, but if you can't develop Common code directly in the submodule, doesn't that make it more difficult to develop? Can someone please explain the proper usage of submodules?

Answer 1

You can develop directly from a submodule (as I explained in "true nature of submodules").
But each time you commit something in the submodule (and publish it somewhere), you need to make sure to commit as well in the parent repository.

Now you have two submodules organizations:

• recursive includes

A
  Common
    (Common could depend itself on other dependencies)
B
  Common
    ...

• or direct includes (direct dependencies only)

ParentA
   A
   Common
   Other A dependencies
ParentB
   B
   Common
   Other B dependencies

(Actually, if Common had dependencies of its own, A or B would depend on "ParentCommon" which would refer to Common and all its direct dependencies)

Unless you have a structural imperative where Common must be a subdirectory of A, I would recommend the second organization, which is closer to your C:\dev\A, C:\dev\Common one.

Actually, I prefer only one depth dependencies, where for ParentA, I list all dependencies, direct and indirect, as submodules.

That is because, with any tool attempting to manage dependencies, you will still need to make decisions on:

• dependencies conflicts (A depends on X which needs Z1, and Y which needs Z2: which version of Z do you want to use/compile in your project?)
• dependencies override (when A depends on X which needs Z1, whereas A also choose to use X2)
• dependencies cycle (where A depends on B which relies on C which uses... A!)

Answer 2

Submodules are not something that works particularly smoothly, the way you intend it here. As you've probably noticed, to commit a change to a project in a submodule (as of Git v1.7, probably in perpetuity), you need to:

1. Make the change in the submodule.
2. Commit to the submodule, getting a new SHA hash.
3. Update the outer project's .gitmodules file with the new hash.
4. Commit to the outer project.

This is cumbersome if you're developing the submodule and the outer project(s) in lockstep. It's "the right way," as far as preferring stable software configurations over simplicity when mixing codebases, but the command sequence is pathologically long in this case.

Popping the why stack, though, lockstep development with submodules probably indicates one of several larger problems:

• You do not have a clear interface between your subcomponent, and implementation details leak across the boundaries.
• You do not have a well-thought interface to your subcomponent, so you keep having to reinvent things as you solve a more interesting problem.
• Your interface is stable and high-quality, but you do not have good QA process on the submodule code, so instead you're constantly fixing that while trying to get other work done.
• (Esp. if A and B keep changing different things) Your "common" code is actually much smaller than you think, or should be divided differently.

None of these cases are necessarily true of you, but these are among the problems that cause the horrible lockstep submodule update workflow you're having. Submodules work pretty well when you could almost rely on shared libraries and header files, but there's some compelling reason (e.g. weird compile flags that must be consistent) to compile from source. My approach would be to fix these problems, if they're the real root causes.

If these problems aren't there, however, it may be that git is the wrong tool for you. It pains me to say it, but this is definitely a possibility.