Keeping git mirrors in sync at all times

Question

I have several sites that use Drupal, I have several servers, live, dev1, dev2...

Drupal's codebase repo is big (112Mb), so I'm keen to make the most of git's hard-linking abilities so that each time I add a site it's not duplicating this.

So on, say, the live server I have a bare master repo, and all my sites are clones of this, each using a different branch. This is great on the one server, hard links are used, it's fast and efficient.

But on my dev servers, they typically all clone from the bare master repo, which means two sites on the same machine can't use hard links to save space.

What I'd like to do is set up a mirror of the bare repo on each of my dev servers, and then clone from that.

dev1$ git clone --mirror live:master-bare-repo  dev1-mirror-repo
dev1$ git clone -b site1 dev1-mirror-repo site1
dev1$ git clone -b site2 dev1-mirror-repo site2

All good so far. But I want the mirrors to stay in sync at all times. So I used post-receive hook on dev1's mirror to do git push --mirror origin. Now if site1 on dev1 pushes commits, they are magically pushed to the master-bare-repo.

But what if I make a change on the live server, and push that? I can't set up a post-receive hook to push to the other(s) because that would presumably trigger their post-receive hooks which would end up in recursion?

Is there some clever way around this?

Answer 1

First of all, you won't end up in a recursion, since the post-receive hook isn't executed when "Everything is up to date" (as noted in this other question), which will be the result of the pushes from the mirrors to the live server.

On the other hand, that's not all of a scalable design (every time you add a new mirror, you'll need to change your live server's hook to add a site to push). You'll probably find more elegant to use a "lazy" sync strategy in your mirrors: when they receive a push they don't just push to the master, but before that they fetch/pull from the master. This way you don't need to setup a hook in the master and synchronization strategy will be completely managed by mirrors. The downside of this strategy is that you might eventually want to make a change to the live server that you want to be propagated to the mirrors before they need to push any change. So you have to ponder whether changes to your masters will be so critical to compensate the trade-off in scalability. Of course, a patch to make this "scalable" design also "synchronizable" is by using an external cron job to check periodically for changes in master, as suggested in comments.