git-svn fails to dcommit, even after clean checkout

Question

I'm trying to use git-svn with the following project:

https://svn.apache.org/repos/asf/commons/sandbox/gsoc/2010/scxml-js/trunk/

Which I've checked out using the standard git svn clone https://svn.apache.org/repos/asf/commons/sandbox/gsoc/2010/scxml-js/trunk/ -T trunk -b branches -t tags invocation.

I started having trouble using git-svn dcommit with git-svn after branching with git checkout -b and not git svn branch, and then merging that local branch back into master, and trying to dcommit. Now, strangely, even after doing a clean checkout, I am still having errors when I attempt to dcommit.

The following rather long trace is representative of my current workflow with git-svn:

jacob@jacob-laptop:~/workspace/gsoc2010/scxml-js$ git svn dcommit
Committing to https://[email protected]/repos/asf/commons/sandbox/gsoc/2010/scxml-js/trunk ...
        M       build-common.xml
        M       src/xslt/backends/js/StateTableStatechartGenerator.xsl
        M       src/xslt/backends/js/SwitchyardStatechartGenerator.xsl
        M       src/xslt/ir-compiler/appendTransitionInformation.xsl
        M       src/xslt/ir-compiler/copyEnumeratedEventTransitions.xsl
        M       src/xslt/ir-compiler/flattenTransitions.xsl
        M       src/xslt/ir-compiler/nameTransitions.xsl
        M       src/xslt/ir-compiler/numberStatesAndTransitions.xsl
        M       src/xslt/ir-compiler/splitTransitionTargets.xsl
        M       src/xslt/util/language-frontend-module-generators/sh-from-topo-sort-list-generator.xsl
        M       src/xslt/util/topo-sort.xsl
Committed r1037491
W: 502073c202cb1fbe089f0e8b7100304f0aa74729 and refs/remotes/trunk differ, using rebase:
:100644 100644 7930fd690be01a5716490704b5556e2b5cab5ba6 51047f2c8cf1daaa94ae44433a9aa2acc222ae43 M      build-common.xml
:040000 040000 59251941b33bda1376881b33bb2263ea87b1b947 2f5e2586e26bc7d05def065cfcf05327982b81d7 M      src
Current branch master is up to date.
# of revisions changed
before:
 39c6f7e2d7b82cc782b8770d1ddb0ecb4f86e780
c5f360feb3a6bdaab2fb2f48d05e8691824e3231

after:
 502073c202cb1fbe089f0e8b7100304f0aa74729
39c6f7e2d7b82cc782b8770d1ddb0ecb4f86e780
c5f360feb3a6bdaab2fb2f48d05e8691824e3231
 If you are attempting to commit  merges, try running:
         git rebase --interactive --preserve-merges  refs/remotes/trunk
Before dcommitting
jacob@jacob-laptop:~/workspace/gsoc2010/scxml-js$ git svn rebase
        M       build-common.xml
        M       src/xslt/ir-compiler/nameTransitions.xsl
        M       src/xslt/ir-compiler/copyEnumeratedEventTransitions.xsl
        M       src/xslt/ir-compiler/appendTransitionInformation.xsl
        M       src/xslt/ir-compiler/flattenTransitions.xsl
        M       src/xslt/ir-compiler/numberStatesAndTransitions.xsl
        M       src/xslt/ir-compiler/splitTransitionTargets.xsl
        M       src/xslt/backends/js/SwitchyardStatechartGenerator.xsl
        M       src/xslt/backends/js/StateTableStatechartGenerator.xsl
        M       src/xslt/util/language-frontend-module-generators/sh-from-topo-sort-list-generator.xsl
        M       src/xslt/util/topo-sort.xsl
r1037491 = e8d820cfdca882a57fbfa003b9673313be6c9621 (refs/remotes/trunk)
First, rewinding head to replay your work on top of it...
Applying: Changed some paths around so python fronend works.
Applying: Removed test_with_xsltproc.sh. This file no longer needed, as we now generate a bash backend module.
jacob@jacob-laptop:~/workspace/gsoc2010/scxml-js$ git svn dcommit
Committing to https://[email protected]/repos/asf/commons/sandbox/gsoc/2010/scxml-js/trunk ...
        M       src/python/scxml/cgf/backends/js
        M       src/python/scxml/cgf/xslt
Committed r1037493
W: 8563b885bfca77ccb33a87f56f78e6323af022a9 and refs/remotes/trunk differ, using rebase:
:040000 040000 c33b3d66ecc6735af3241d7f9cee383d2736f614 59251941b33bda1376881b33bb2263ea87b1b947 M      src
Current branch master is up to date.
# of revisions changed
before:
 574e2e495fa12cf3d81004638a06e3315d0abba5

after:
 8563b885bfca77ccb33a87f56f78e6323af022a9
574e2e495fa12cf3d81004638a06e3315d0abba5
 If you are attempting to commit  merges, try running:
         git rebase --interactive --preserve-merges  refs/remotes/trunk
Before dcommitting

I basically need to do a git svn dcommit followed by a git svn rebase for every commit since the last dcommit. This usually works cleanly, but sometimes fails and requires more manual work.

What could be causing this problem with dcommit, such that even using a clean checkout does not resolve it?

Answer 1

I just ran into this and finally figured out what's going on: When you access an SVN repository via HTTP and write-through proxying, the recommended post-commit hook runs svnsync in the background. The commit via HTTP finishes as soon as the SVN server is done, but before the HTTP proxy you're reading through has been updated to the new revision. git-svn does a commit quickly followed by a fetch and sees the stale HTTP proxy without the new revision. It assumes it got the new revision and diffs the tip of the remote with what it tried to commit and sees the conflict.

If you control the SVN proxy configuration you can make the svnsync synchronous (this worked for me for a long time until someone surprised me with a config change). Otherwise I think git-svn needs a retry mechanism:

diff --git a/git-svn.perl b/git-svn.perl
index 09c4ca5..af9aea1 100755
--- a/git-svn.perl
+++ b/git-svn.perl
@@ -57,6 +57,7 @@ use File::Spec;
 use File::Find;
 use Getopt::Long qw/:config gnu_getopt no_ignore_case auto_abbrev/;
 use IPC::Open3;
+use Time::HiRes qw/usleep/;
 use Git;

 BEGIN {
@@ -574,7 +575,17 @@ sub cmd_dcommit {
                                $gs->{inject_parents_dcommit}->{$cmt_rev} =
                                                               $parents->{$d};
                        }
-                       $_fetch_all ? $gs->fetch_all : $gs->fetch;
+                       my $retry;
+                       fetch: for ($retry = 0; $retry < 30; ++$retry) {
+                           $_fetch_all ? $gs->fetch_all : $gs->fetch;
+                           last fetch if ($gs->rev_map_max >= $cmt_rev);
+                           # Asynchronous commit push not complete
+                           usleep(20000 * ($retry + 1));
+                       }
+                       if ($retry > 0 && $gs->rev_map_max < $cmt_rev) {
+                           fatal "New revision $cmt_rev did not appear",
+                                 "in repository after $retry retries.";
+                       }
                        $last_rev = $cmt_rev;
                        next if $_no_rebase;

$

If this still seems like a good idea tonight I'll submit a patch.

Update: While using this I ended up with:

Author: syncuser <syncuser@d3ff0b4f-3c2c-0410-a809-ba59474314df>

on one of my own commits. So there may still be a race condition where the script sees a partial sync. This is annoying but not fatal.