Merge error after converting Git submodule to subtree

Question

I have a project where I was originally using submodules for some dependent code. It turns out that submodules are not really appropriate for this project (and they are hard to use in practice), so I am converting each submodule to a subtree (using the new git-subtree feature).

In my working repository, I have successfully removed each submodule and have added the old submodule repo as a subtree. No problem with this.

When I go to another clone and attempt to pull from the first one, I get the following error from the merge step:

error: The following untracked working tree files would be overwritten by merge:
        sub/.gitignore
        sub/Makefile
        sub/README
        sub/src/main.c
        ... and so on for all files in sub/
Aborting

It seems that this is because the files in sub/ never really existed in the main repository in the first place, and when Git applies the patch to update .gitmodules it doesn't remove the directory with the submodule files. When processing the next commit, where Git tries to create the new files in sub/ that are now part of the main repository, all those files conflict with the still-existing files in sub/.

The workaround I have found is to use rm -rf sub before git pull, which avoids this problem.

My question is, is there any command line switch I can use with git merge that says "Overwrite any files that happen to exist in the working directory"? Even better would be a feature where git merge would look at the contents of the existing file, and if the contents are identical to the file it was going to create anyway, suppress the error message and continue.

UPDATE: I have created Git repositories that demonstrate this problem, to show exactly what I'm talking about. To reproduce:

$ git clone https://github.com/ghewgill/q14224966.git
$ cd q14224966
$ git submodule init
$ git submodule update
$ git merge origin/branch

This should result in the error message

error: The following untracked working tree files would be overwritten by merge:
    sub/Makefile
    sub/README
    sub/src/main.c
Please move or remove them before you can merge.
Aborting

Answer 1

I know your question was specific to merge, but I've had similar problems with merging git submodules. I think this solution will work with your problem, even if it's not directly addressing the merge question.

I've found that by forcibly checking out the branch you want to merge, then going back to master, everything works out with submodules.

To get things working in your example:

$ git clone https://github.com/ghewgill/q14224966.git
$ cd q14224966
$ git submodule init
$ git submodule update
$ git checkout -f origin/branch
$ git checkout master
$ git merge origin/branch

This works because it's basically doing your rm -rf step for you. Granted, this is a bit roundabout, and maybe not worth doing if you only have the one submodule like your example does. But I've found it to be quite the timesaver when working in a project with many submodules.

Also, as has been pointed out in the comments, if you want to avoid making changes to the work tree, you can use this:

$ git clone https://github.com/ghewgill/q14224966.git
$ cd q14224966
$ git submodule init
$ git submodule update
$ git reset origin/branch
$ git reset --hard master

This works in roughly the same way, but avoids checking out other files in the process. I haven't had a chance to use this in the wild, but it seems like a sound method.

There's also $ git merge -s subtree origin/branch. It works with your example, but I've had unexpected results with it when more than one submodule is involved. You might have better luck, though.

Answer 2

You can't get git-merge (or any other command) to forcibly clobber files it doesn't think it knows about, no. git tries pretty hard not to do anything totally irreversible.

But with many submodules, you can make your deletion a bit easier and safer with git submodule foreach:

$ git submodule foreach 'rm -rf $toplevel/$path'
Entering 'sub'
$ git merge origin/branch
Updating a231acd..6b4d2f4
Fast-forward
...

Answer 3

(Caveat: I've never worked with subtrees and I don't know how complicated your actual repo is, so these solutions may not actually work for you.)

From playing around with your sample repo, I've found two solutions that both seems to work, though they produce different commit trees:

1. Use git merge -s resolve origin/branch

   ~/q14224966[master]> git reset --hard origin/master
   HEAD is now at a231acd add submodule
   ~/q14224966[master]> touch other.c && git add . && git commit -m "New commit."
   [master bc771ac] New commit.
    0 files changed
    create mode 100644 other.c
   ~/q14224966[master]> git merge -s resolve origin/branch 
   Trying really trivial in-index merge...
   error: Merge requires file-level merging
   Nope.
   Trying simple merge.
   Simple merge failed, trying Automatic merge.
   Adding sub/Makefile
   Adding sub/README
   Adding sub/src/main.c
   Merge made by the 'resolve' strategy.
    .gitmodules    | 3 ---
    sub            | 1 -
    sub/Makefile   | 1 +
    sub/README     | 1 +
    sub/src/main.c | 1 +
    5 files changed, 3 insertions(+), 4 deletions(-)
    delete mode 160000 sub
    create mode 100644 sub/Makefile
    create mode 100644 sub/README
    create mode 100644 sub/src/main.c
   ~/q14224966[master]> ls
   README   main.c   other.c  sub/
   ~/q14224966[master]> cd sub/
   ~/q14224966/sub[master]> ls
   Makefile  README    src/
   ~/q14224966/sub[master]> git status
   # On branch master
   # Your branch is ahead of 'origin/master' by 5 commits.
   #
   nothing to commit (working directory clean)
   ~/q14224966/sub[master]> cd ..
   ~/q14224966[master]> git status
   # On branch master
   # Your branch is ahead of 'origin/master' by 5 commits.
   #
   nothing to commit (working directory clean)
   

   Here's the resulting commit tree:

2. Use a rebase instead of a merge:

   ~/q14224966[master]> git reset --hard origin/master 
   HEAD is now at a231acd add submodule
   ~/q14224966[master]> touch other.c && git add . && git commit -m "New commit."
   [master ae66060] New commit.
    0 files changed
    create mode 100644 other.c
   ~/q14224966[master]> git rebase origin/branch 
   First, rewinding head to replay your work on top of it...
   Applying: New commit.
   ~/q14224966[master]> ls
   README   main.c   other.c  sub/
   ~/q14224966[master]> cd sub/
   ~/q14224966/sub[master]> ls
   Makefile  README    src/
   ~/q14224966/sub[master]> git status
   # On branch master
   # Your branch is ahead of 'origin/master' by 4 commits.
   #
   nothing to commit (working directory clean)
   ~/q14224966/sub[master]> cd ..
   ~/q14224966[master]> git status
   # On branch master
   # Your branch is ahead of 'origin/master' by 4 commits.
   #
   nothing to commit (working directory clean)
   

   Here's the resulting commit tree: