Why are Mercurial backouts in one branch affecting other branches?

Question

This is a difficult situation to explain, so bear with me. I have a Mercurial repository with 2 main branches, default and dev.

Work is usually done in a named branch off of dev (a feature branch). There may be many feature branches at any one time. Once work is completed in that branch, it is merged back into dev.

When the time comes to prepare a release, another named branch is created off of dev (a release branch). Sometimes it is necessary to exclude entire features from a release. If that is the case, the merge changeset from where the feature branch was merged into dev is backed out of the new release branch.

Once a release branch is ready to be released, it is merged into default (so default always represents the state of the code in production). Work continues as normal on the dev branch and feature branches.

The problem occurs when the time comes to do another release, including the feature that was backed out in the previous release. A new release branch is created as normal (off of dev). This new release branch now contains the feature that was backed out of the previous release branch (since the backout was performed on the release branch, and the merge changeset remains on the dev branch).

This time, when the release branch is ready for release and is merged into default, any changes that were backed out as a result of the merge backout in the previous release branch are not merged into default. Why is this the case? Since the new release branch contains all of the feature branch changesets (nothing has been backed out), why does the default branch not receive all of these changesets too?

If all of the above is difficult to follow, here's a screenshot from TortoiseHg that shows the basic problem. "branch1" and "branch2" are feature branches, "release" and "release2" are the release branches:

Answer 1

I believe the problem is that merges work differently than you think. You write

Since the new release branch contains all of the feature branch changesets (nothing has been backed out), why does the default branch not receive all of these changesets too?

When you merge two branches, it's wrong to think of it as applying all changes from one branch onto another branch. So the default branch does not "receive" any changesets from release2. I know this is how we normally think of merges, but it's inaccurate.

What really happens when you merge two changesets is the following:

1. Mercurial finds the common ancestor for the two changesets.

2. For each file that differ between the two changesets Mercurial runs a three-way merge algorithm using the ancestor file, the file in the first changeset and the file in the second changeset.

In your case, you are merging revision 11 and 12. The least common ancestor is revision 8. This means that Mercurial will run a three-way merge between files from there revisions:

• Revision 8: no backout

• Revision 11: feature branch has been backed out

• Revision 12: no backout

In a three-way merge, a change always trumps no change. Mercurial sees that the files have been changed between 8 and 11 and it sees no change between 8 and 12. So it uses the changed version from revision 11 in the merge. This applies for any three-way merge algorithm. The full merge table looks like this where old, new, ... are the content of matching hunks in the three files:

ancestor  local  other -> merge
old       old    old      old (nobody changed the hunk)
old       old    new      new (they changed the hunk)
old       new    old      new (you changed the hunk)
old       new    new      new (hunk was cherry picked onto both branches)
old       foo    bar      <!> (conflict, both changed hunk but differently)

I'm afraid that a merge changeset shouldn't be backed out at all because of this surprising merge behavior. Mercurial 2.0 and later will abort and complain if you try to backout a merge.

In general, one can say that the three-way merge algorithm assumes that all change is good. So if you merge branch1 into dev and then later undo the merge with a backout, then the merge algorithm will think that the state is "better" than before. This means that you cannot just re-merge branch1 into dev at a later point to get the backed-out changes back.

What you can do is to use a "dummy merge" when you merge into default. You simply merge and always keep the changes from the release branch you're merging into default:

$ hg update default
$ hg merge release2 --tool internal:other -y
$ hg revert --all --rev release2
$ hg commit -m "Release 2 is the new default"

That will side-step the problem and force default be just like release2. This assumes that absolutely no changes are made on default without being merged into a release branch.

If you must be able to make releases with skipped features, then the "right" way is to not merge those features at all. Merging is a strong commitment: you tell Mercurial that the merge changeset now has all the good stuff from both its ancestors. As long as Mercurial wont let you pick your own base revision when merging, the three-way merge algorithm wont let you change your mind about a backout.

What you can do, however, is to backout the backout. This means that you re-introduce the changes from your feature branch onto your release branch. So you start with a graph like

release: ... o --- o --- m1 --- m2
                        /      /
feature-A:   ... o --- o      /
                             /
feature-B:  ... o --- o --- o 

You now decided that the A feature was bad and you backout the merge:

release: ... o --- o --- m1 --- m2 --- b1
                        /      /
feature-A:   ... o --- o      /
                             /
feature-B:  ... o --- o --- o 

You then merge another feature into your release branch:

release: ... o --- o --- m1 --- m2 --- b1 --- m3
                        /      /             /
feature-A:   ... o --- o      /             /
                             /             /
feature-B:  ... o --- o --- o             /
                                         /
feature-C:  ... o --- o --- o --- o --- o 

If you now want to re-introduce the A feature, then you can backout b1:

release: ... o --- o --- m1 --- m2 --- b1 --- m3 --- b2
                        /      /             /
feature-A:   ... o --- o      /             /
                             /             /
feature-B:  ... o --- o --- o             /
                                         /
feature-C:  ... o --- o --- o --- o --- o 

We can add the deltas to the graph to better show what changes where and when:

                     +A     +B     -A     +C     --A
release: ... o --- o --- m1 --- m2 --- b1 --- m3 --- b2

After this second backout, you can merge again with feature-A in case new changesets have been added there. The graph you're merging looks like:

release: ... o --- o --- m1 --- m2 --- b1 --- m3 --- b2
                        /      /             /
feature-A:   ... o -- a1 - a2 /             /
                             /             /
feature-B:  ... o --- o --- o             /
                                         /
feature-C:  ... o --- o --- o --- o --- o 

and you merge a2 and b2. The common ancestor will be a1. This means that the only changes you'll need to consider in the three-way merge are those between a1 and a2 and a1 and b2. Here b2 already have the bulk of the changes "in" a2 so the merge will be small.