Git (Process): Use Rebase or Double QA?

Question

Our shop has a fairly rapid deploy cycle (1-2 weeks between when the code is written and when it's released). As such, after a bit of experimentation we've adopted the following pattern for our Git usage:

1. Developer creates a branch, based off master, for their specific ticket (ie. "bug-5555" branch)
2. Developer commits the code for the fix to that branch
3. A supervisor rebases our "pre-master" branch (a copy of master with the current release candidates on top of it) in to the bug branch
4. A supervisor merges (fast-forward-style) the commits from the bug branch in to the pre-master branch
5. The QA team qa's the fixes in the pre-master branch
6. If a fix fails QA, it is rebased out of the pre-master branch
7. (After a fix for the QA failure is completed, steps 3-5 are repeated)
8. When we're ready to release, the pre-master branch becomes the new master branch

This style has a number of advantages for us: it makes it so the QA team is testing exactly what is going to go live, it makes it easy for us to put stuff in and take stuff out of QA, it keeps the code for a specific fix all nice and tidy in its own branch, etc.

However, several people here are concerned about our use of rebase (both to rebase the current pre-master in to the fix branch before merging that fix branch in to pre-master, and to rebase the pre-master branch to take out failed fixes). Their concern is that rebasing could potentially result in lost history, and as such we should rebase as infrequently as possible.

However, without rebasing the only alternative we've come up with is to make the pre-master branch a dead-end branch, used only for QA. This would make rebasing that branch safer, and when we're ready to release we'd re-merge the fix branches directly in to master. The problem with this approach though is that QA wouldn't actually be testing what's going live, and an improper merge conflict resolution (when merging the fixes in to master) could easily slip by them (unless they re-qa everything all over again). This is the main reason why we've stuck with our current approach, despite the rebasing concerns.

So, with that very long prelude out of the way, my actual question is ... well it's two-part:

1. Which do you think is worse: the risk of not QA-ing exactly what is going live, or the risk of losing history (or actual code) from the (limited) amount of rebasing we do?
2. Does anyone see a third option to what I've outlined, which would give us the same flexibility and testing of what's actually going to go live, without the dangers of rebasing?

Answer 1

In general, I would agree that without a good mechanism for keeping track of things that are in flight (which could be outside your SCM system itself, e.g., in your bug tracker), rebasing things as a rule could be dangerous.

There are (at least) 2+ very good resources on how to manage a release process using Git branches.

• gitworkflows(7) describes how git developers manage their own release processes.
• Junio Hamano (the git maintainer) expands on this in his own description of how he handles integrating incoming features: maintain-git.txt.
• In "A successful Git branching model", Vincent Driessen describes a slightly different model.

Git itself uses a "proposed updates" branch (called pu) that mirrors your pre-master branch. This branch consists of merges from various fix branches. This branch is intended for really unstable features that need to be tested and integrated. It can relatively freely be rebased and reset. (Again, each individual topic/feature branch is tracked outside of Git). Features are generally only merged once into pu and pu gets reset against something more stable very regularly.

More stable changes get merged into next for wider testing. Again, this is handled with merging, not rebasing. Your pre-master branch serves similar purpose to both next and pu. A feature might get merged multiple times into next, to incorporate additional feedback. Development still happens on the topic branch though.

When the topic branch is deemed stable enough, it is merged to master.

To help track what is going on, Git has a concept of "What's Cooking". Junio Hamano, the Git maintainer, has a script called cook that helps keep things straight for everyone and what state various topic branches are in.

Of course, there is no explicit QA process for Git; in your case, with a real QA team, you can do some combination of things.

• The rebases against master in your step (3) are okay. They are a reasonable part of development.
• Instead of merging fast-forward style, do as Driessen suggests and use --no-ff to explicitly track the branch provenance.
• For handling QA detected failures, re-create the pre-master branch by resetting and re-merging, or by reverting the merge commit, or by merging in a new fix that appeared on top of the old fix branch.
• When you're ready to release to master, create a new integration branch that directly merges in all of the successful fixes just once from each fix branch. You can verify that this tree is literally the code that QA tested (e.g., git diff integration pre-master is empty). Then merge the integration branch into master (again, using --no-ff to track who did it and when).
• For the next cycle, you can re-create pre-master from scratch by doing a git checkout pre-master; git reset --hard master, or you can do a presumably trivial merge back from master (resolving all conflicts in favor of master, e.g. git checkout pre-master; git merge -s recursive -X theirs master).
• You can, if you want, use tags to track specific releases to QA along pre-master.

The differences here are mostly just to use the merges to help you keep track of what has been merged and when and by whom. The other nice thing about merges is that developers (and QA) will see fewer forced updates, and more just regular updates.

Again, the real thing to emphasize is that there must be something that is used to track what has been merged (or rebased) into a stable branch and tested successfully.