Can I use git submodule within the same repository?

Question

In my project there is a folder with common tools that I want to include in each of my app components. I just want one master version of it and I want the rest of the copies to be updated with git commands as normal. Is there a way to do this while keeping everything in the same repository? I know got submodule works with multiple repositories but it doesn’t seem like it works with just a single repository. Does anyone have any insight on the matter?

Answer 1

You may want to look into git subtree. Have a look at these two articles :

• Git subtree: the alternative to Git submodule
• The power of Git subtree

Its documentation is not served on git-scm.org (I couldn't say why), you can read it by typing git help subtree, or by reading it from the source code in git's repo :

• git-subtree (version 2.29 - Q4 2020)

---

Technically, you can include a repo within itself as a submodule ; the unusual stuff would come from :

• the commits that would be checked out at the submodule's directory,
• the branch names,
• the history of the commts for the submodule.

Suppose your code looks like this :

Readme.md
libs/util/fileA.py
libs/util/fileB.py
src/main.py

and you would like to have the content of libs/util/ as a "self hosted" submodule.

1. about the commits for the submodule :

This would mean that the complete content of a certain commit of your repo should be :

fileA.py
fileB.py

You would have, in the same repo, commits that represent "the full project" and commits that represent "just the util lib" sitting one next to another.
This is perfectly manageable, just surprising at first.

2. about the branch names :

since your repo probably already has a branch named master which represents the state of your "complete" project, you can't use that name for a branch of the "submodule" part.

The simplest way would be to choose a set of names dedicated to that submodule (for example : util/*) and stick to it.

3. about the history :

It wouldn't make much sense to have in the same branch commits for the complete repo and commits for the submodule only. The branches dedicated to the submodule would have no relation with the branches of the "complete" project.

Again : git can perfectly manage that. Here is one way to create such a starting branch in your repo :

# from your repo's 'master' branch :
git checkout -b util/master
git filter-branch --subdirectory-filter  libs/util

---

When you look at how git-subtree works, it allows you to handle the same kind of setup : unrelated branches stored within the same repo, and checking out a sub branch at a location you choose.

Its main advantage is : since it was intended to work that way from the beginning, the commands will probably give you less friction, and less way to mess things up in your repo.

Answer 2

Experimenting a little, the answer seems to be "sort-of". The command supports it with something like

git submodule add -b libfoo -- ./ libs/foo

where libfoo is the name of the branch that the submodule exists in, ./ is a relative URL to the "upstream repository", and libs/foo is the "working tree" of the submodule checkout (and also the default name of the submodule; you may want to override this with --name libfoo).

Digging around a little, it doesn't quite do what you might want:

• It seems to do a ~full checkout (with all the remote tracking branches, not just the submodule branch).
  • You can probably fix this by editing .git/modules/libs/foo/config.
• It does not share storage with the "enclosing" repo.
  • You can probably make it share storage with symlinks (as git-new-workdir does).
  • I'm less sure about making this share storage in the same way as git worktree. Maybe this is just a matter of pointing .git/modules/libs/foo/commondir at the enclosing repo?
• If you have an origin (a.k.a. upstream?) set, I think the branch must already exist on the origin repo.
• If you don't have an origin set, it uses the absolute path of the local repo as the origin "URL" of the submodule (see .git/modules/libs/foo/config). This will break if you move your checkout somewhere else on the filesystem.

git-subtrac (from the author of git-subtree) is designed around the "submodules in the same repo" idea, but I'm not sure how much it helps with the above.