Using a git subrepo which has ANOTHER git subrepo in a mercurial repository, is this possible?

Question

I have a mercurial repository, and have added without problems a git subrepo (hg 1.8).

Problem is: this git subrepo has ANOTHER git subrepository inside itself and it isn't being pulled (it's in the git's subrepo .gitmodules file), unless I do a git clone --recursive on my git subrepo: doing this way it works.

The problem: I do a hg pull in my repository in another machine, it pulls the git subrepo, but it doesn't pull the .gitmodules. The .gitmodules was only pulled in the other machine when I did a git clone --recursive.

Does anybody has any suggestions to handle this situation? The ugly solution is to do a git clone and simply add all the files (including git metadata) to my mercurial repository, without acting like a subrepo.

Answer 1

I suppose the best fix would be to patch Mercurial’s Git subrepository support to always use Git’s recursive options (e.g. git clone --recursive when cloning a Git-based subrepository, git pull --recurse-submodules && git submodule update after pulling an updated Git-based subrepository, etc.). I know that the Git developers specifically chose to not automatically initialize submodules because one of the workflows they want to support is “I never want to see any of the the submodules”, but maybe “always initialize all subrepositories” is a better match to the default Mercurial mode of operation (I am not much of a Mercurial user, so I do not have a good idea of what the default Mercurial style would be).

---

Until that happens, you might be able to work around the problem by translating the subrepo/.gitmodules entries into .hgsub entries. It is easy to do manually, but you could probably automate it if it was important (use git config to extract the paths and URLs from .git/config and/or .gitmodules). This may be unappealing if you are dealing with a .gitmodules file that changes much (you would have to be very diligent about synchronizing .hgsub each time .gitmodules changed).

I tested this with four repositories:

• gitsub — a “leaf” repository (no Git submodules)
• gitsuper — a Git “superproject”;
  gitsub/ is gitsub as a submodule
• hgsuper2 — a Mercurial “superproject”;
  gitsuper/ is gitsuper as a subrepository,
  gitsuper/gitsub is gitsub as a subrepository.
• hgsuper2-clone — a cloned Mercurial “superproject”;
  gitsuper/ is gitsuper as a subrepository,
  gitsuper/gitsub is gitsub as a subrepository.

I built and tested them like this:

1. Create gitsub. Add and commit some content.
2. Create gitsuper.
   1. Add some content.
   2. git submodule add url-of-gitsub gitsub && git submodule init
   3. git commit -m 'added gitsub'
3. Create hgsuper2.
   1. Add some content.
   2. git clone --recursive url-of-gitsuper gitsuper
   3. echo 'gitsuper = [git]url-of-gitsuper' >> .hgsub
   4. echo 'gitsuper/gitsub = [git]url-of-gitsub' >> .hgsub
      These last two steps could be automated from bits of gitsuper/.git/config and gitsuper/.gitmodules.
   5. hg add .hgsub && hg commit -m 'added Git subrepositories'
4. Clone hgsuper2-clone from hgsuper2.
   It gets the appropriate contents in gitsuper/ and gitsuper/gitsub/.
5. Update and commit new content to gitsub.
6. Update gitsuper.
   1. Add or change some content and stage it.
   2. (cd gitsub && git pull origin master)
   3. git add gitsub && git commit -m 'updated gitsuper content (also gitsub)'
7. In hgsuper2, pull changes from Git suprepositories.
   1. (cd gitsuper && git pull --recurse-submodules && git submodule update)
      The content in gitsuper/ and gitsuper/gitsub/ is updated by the pull.
   2. hg commit -m 'updated gitsuper (and its contents)'
8. Pull into hgsuper2-clone.
   1. hg pull -u
      The content from Git has been updated.

My tests worked (using Mercurial 1.8.1 and Git 1.7.4.1), but I noticed one bug. Mercurial creates and checks out an oddly named Git branch (origin/master (i.e. refs/heads/origin/master) instead of using a detached HEAD (like Git does with its submodules) or just using master (i.e. refs/heads/master)). It also seems to get a bit wedged at times, resulting in errors like this:

fatal: git checkout: branch origin/master already exists
abort: git checkout error 128 in gitsuper

I worked around the problem by going into the Git repository in question (the Git-based Mercurial subrepository) and deleting the branch with git checkout HEAD~0 && git branch -D origin/master (the first detaches HEAD and (more importantly) moves off of the branch so it can be deleted by the next command). This workaround is completely safe as long as you do not have any local changes changes in the Git repository.

Another small problem is that you will need to run git submodule init to let Git know about its submodules before issuing Git submodule commands in a Git super repository that was created by Mercurial (the submodules were cloned to the right places, but they were established by Mercurial, so there are no entries for them in .git/config).

Similarly, if you plan on authoring changes to the content that is managed by Git from inside the Git-based Mercurial subrepository, then you should be careful to always add any Git submodules, commit, and push from the Git subrepositories before committing in the Mercurial “superproject”. Otherwise, you might end up with a situation where Mercurial uses one combination of gitsuper and gitsub while gitsuper itself refers to a different version of gitsub. In other words, since you will be bypassing Git’s submodule code (by managing the Git submodules as Mercurial subrepositories), you will need to be careful to keep Git’s view of the submodules synchronized with that of Mercurial.