Empty commits removed after interactive rebase, even though --keep-empty is used

Question

I have some trouble using the --keep-empty option of git rebase, and I'm not sure whether I'm misunderstanding what this option does, or there's a bug.

Here is a minimal example:

Setup

1. Create a new Git repository and an initial, unrelated commit.

   $ git init
   $ echo something >base.txt
   $ git add base.txt
   $ git commit -m 'some base commit to not run into the root corner case'
   

2. Create a new commit which adds two new files.

   $ echo A >a.txt; echo B >b.txt
   $ git add a.txt b.txt
   $ git commit -m 'add A and B'
   

3. Modify one of the files.

   $ echo A1 >a.txt
   $ git add a.txt
   $ git commit -m 'change A'
   

4. Modify the other file.

   $ echo B1 >b.txt
   $ git add b.txt
   $ git commit -m 'change B'
   

Rebase

$ git checkout -b rebased master
$ git rebase --keep-empty -i :/base

… choosing to edit the commit where A and B are added, and changing it so that only B is added (in a real scenario the reason might be that A is confidential):

$ git rm a.txt
$ git commit --amend
$ git rebase --continue

Naturally, the next commit where A is modified now gives a conflict:

error: could not apply 182aaa1... change A

When you have resolved this problem, run "git rebase --continue".
If you prefer to skip this patch, run "git rebase --skip" instead.
To check out the original branch and stop rebasing, run "git rebase --abort".
Could not apply 182aaa1701ad100fc02a5d5500cacebdd317a24b... change A

… choosing to not add the modified version of a.txt:

$ git mergetool
Merging:
a.txt

Deleted merge conflict for 'a.txt':
  {local}: deleted
  {remote}: modified file
Use (m)odified or (d)eleted file, or (a)bort? d

The commit where A was modified is now empty:

$ git diff --cached
# nothing

… and finishing the rebase:

$ git rebase --continue
Successfully rebased and updated refs/heads/rebased.

Question

So now I have two versions of my history, with the difference that there is no trace of A in one of them. However, because I chose the --keep-empty option, I still expect an empty commit to exist in rebased, which would show me that A would have been modified, had it been there.

But apparently, this is not the case:

$ git log --oneline master
f893569 change B
182aaa1 change A
3340b71 add A and B
38cb5da some base commit to not run into the root corner case

$ git log --oneline rebased
73a2c05 change B
55f502b add A and B
38cb5da some base commit to not run into the root corner case

Is this not what --keep-empty is supposed to do, or does it not work correctly?

---

Related: Rebase on the root and keep empty commits is a very similar question, but it involves the --root corner case which I explicitly avoided here. And it has no answer, only some comments which suggest that what I'm showing here should work. Another difference is that in the other question the commit is empty in the first place, while here it only becomes empty after resolving a conflict.

Answer 1

It's sort of a bug, due to something that is sort of a feature. :-)

When you run interactive rebase and it "pauses", in reality, it finishes, but leaves some files around to let a new git rebase realize that it's more of a continuation after all. This is fine as far as it goes; you will need to run git rebase --continue later to start a new rebase and tell it: You're not really new, go read the state and act like you're continuing the original rebase.

And, let's look at an "interactive rebase". In reality this is mostly a series of cherry-pick operations: the pick command literally instructs the old rebase shell script—which is being phased out now—to run git cherry-pick.

OK, no big deal so far. But let's consider why an interactive rebase stops. There are two reasons:

1. You marked a commit "edit". It actually commits the cherry-pick, and stops to let you amend the commit or otherwise fuss with it.

2. Or, there was a problem—such as a merge conflict—that forced the stop.

In case (1), when you run git rebase --continue, Git should not make its own commit.

In case (2), when you run git rebase --continue, Git should make its own commit. That is, it should unless—this is the feature part—you make your own commit first. In that case, for case (2) Git should not make its own commit.

Git could, and perhaps should, record the reason-for-stoppage so as to tell these two cases apart ... but it doesn't. Instead, it just looks at the state on --continue.

For a non-interactive rebase, Git knows that it only stops on conflicts, so it knows to try to make a commit, and complain if there is nothing to commit. This is where the --keep-empty or -k flag is useful. (Internally, the non-interactive case uses git format-patch and git am by default, although you can force it to use the interactive machinery with --preserve-merges for instance. I mention this here as it's an implementational reason that Git has to know whether you're being "interactive": as so often happens, here Git lets the implementation dictate the behavior. If Git didn't need this distinction, a --continue could just use the same code for interactive and non-interactive rebase, but Git does need the distinction, and hence doesn't use the same code.)

For an interactive rebase, though, Git allows you to make your own commit in case (2), just before running git rebase --continue (this is the Feature part). If so, the --continue step should just move on to the next commit. So --continue just checks whether there's something to commit now, rather than whether the earlier interactive rebase exited for case (1) vs case (2). This simple implementation trick enables the feature, but also means that --keep-empty cannot work here: Git just doesn't know the difference.

The workaround is to do your own git commit --allow-empty after resolving your merge. In other words, convert case (2) into a simulated case (1), using the "you may make your own commit" feature.

Answer 2

However, because I chose the --keep-empty option, I still expect an empty commit to exist in rebased, which would show me that A would have been modified, had it been there.

But apparently, this is not the case:

Double-check that with Git 2.18 (Q2 2018), considering "git rebase --keep-empty" still removed an empty commit if the other side contained an empty commit (due to the "does an equivalent patch exist already?" check), which has been corrected.

See commit 3d94616, commit 76ea235, commit bb2ac4f (20 Mar 2018) by Phillip Wood (phillipwood).
^{(Merged by Junio C Hamano -- gitster -- in commit d892bee, 25 Apr 2018)}

rebase -i --keep-empty: don't prune empty commits

If there are empty commits on the left hand side of $upstream...HEAD then the empty commits on the right hand side that we want to keep are pruned by --cherry-pick.
Fix this by using --cherry-mark instead of --cherry-pick and keeping the commits that are empty or are not marked as cherry-picks.

And:

rebase --keep-empty: always use interactive rebase

rebase --merge accepts --keep-empty but just ignores it, by using an implicit interactive rebase the user still gets the rename detection of a merge based rebase but with with --keep-empty support.

If rebase --keep-empty without --interactive or --merge stops for the user to resolve merge conflicts then 'git rebase --continue' will fail. This is because it uses a different code path that does not create $git_dir/rebase-apply.
As rebase --keep-empty was implemented using cherry-pick it has never supported the am options and now that interactive rebases support --signoff there is no loss of functionality by using an implicit interactive rebase.

---

Note: this is part of a larger feature added to git rebase in Git 2.18:
See "What exactly does Git's “rebase --preserve-merges” do (and why?)".
With git --rebase-merges (which will ultimately replace the old git --preserve-merges), you now can rebase a whole topology of commit graph elsewhere.

---

With Git 2.27 (Q2 2020), "git rebase" (again) learns to honor "--no-keep-empty", which lets the user to discard commits that are empty from the beginning (as opposed to the ones that become empty because of rebasing).

The interactive rebase also marks commits that are empty in the todo.

See commit 50ed761, commit b9cbd29, commit 1b5735f (11 Apr 2020) by Elijah Newren (newren).
^{(Merged by Junio C Hamano -- gitster -- in commit c7d8f69, 22 Apr 2020)}

rebase: reinstate --no-keep-empty

^{Reported-by: Bryan Turner
Reported-by: Sami Boukortt
Signed-off-by: Elijah Newren}

Commit d48e5e21da ("rebase (interactive-backend): make --keep-empty the default", 2020-02-15, Git v2.26.0-rc0 -- merge listed in batch #8) turned --keep-empty (for keeping commits which start empty) into the default.

The logic underpinning that commit was:

1. 'git commit' errors out on the creation of empty commits without an override flag
2. Once someone determines that the override is worthwhile, it's annoying and/or harmful to required them to take extra steps in order to keep such commits around (and to repeat such steps with every rebase).

While the logic on which the decision was made is sound, the result was a bit of an overcorrection.

Instead of jumping to having --keep-empty being the default, it jumped to making --keep-empty the only available behavior.

There was a simple workaround, though, which was thought to be good enough at the time.

People could still drop commits which started empty the same way the could drop any commits: by firing up an interactive rebase and picking out the commits they didn't want from the list.

However, there are cases where external tools might create enough empty commits that picking all of them out is painful.

As such, having a flag to automatically remove start-empty commits may be beneficial.

Provide users a way to drop commits which start empty using a flag that existed for years: --no-keep-empty.

Interpret --keep-empty as countermanding any previous --no-keep-empty, but otherwise leaving --keep-empty as the default.

This might lead to some slight weirdness since commands like:

git rebase --empty=drop --keep-empty
git rebase --empty=keep --no-keep-empty

look really weird despite making perfect sense (the first will drop commits which become empty, but keep commits that started empty; the second will keep commits which become empty, but drop commits which started empty).

However, --no-keep-empty was named years ago and we are predominantly keeping it for backward compatibility; also we suspect it will only be used rarely since folks already have a simple way to drop commits they don't want with an interactive rebase.