How to set up subgit to mirror an svn repo that looks like a Windows Explorer hierarchy?

Question

Being windows users, we created one svn repo with a hierarchy of folders. The bottom nodes contain the svn standard layout:

ProjectA/
    ApplicationOne/
        ModuleX/
            trunk/
            branches/
            tags/
    ApplicationTwo/
        ModuleY/
            trunk/
            branches/
            tags/

... and so on ad infinitum. The repo now contains around 100+ real svn projects with the trunk/branches/tags structure, but almost none of them at the top level.

How would I configure subgit to handle this?

Answer 1

SubGit can work in two different modes: local mirror mode and remote mirror mode. Below you can find a general overview of these modes and some recommendations for your particular case.

• Local Mirror Mode

  In this mode both Subversion and Git repositories reside on the same host, so SubGit has local access to both SVN and Git sides.

  Below I've provided basic instructions. Please find detailed documentation and common pitfalls in SubGit 'Local Mode' Book.

  1. Configuration

     subgit configure <SVN_REPO>
     SubGit version <VERSION> build #<BUILD_NUMBER>
     
     Detecting paths eligible for translation... done.
     Subversion to Git mapping has been configured:
       /ProjectA/ApplicationOne/ModuleX : <SVN_REPO>/git/ProjectA/ApplicationOne/ModuleX.git
       /ProjectA/ApplicationTwo/ModuleY : <SVN_REPO>/git/ProjectA/ApplicationTwo/ModuleY.git
       ...
     
     CONFIGURATION SUCCESSFUL
     ...
     

     This command tries to auto-detect repository layout and generate configuration file at <SVN_REPO>/conf/subgit.conf. It may take a while in case of big Subversion repository like yours.

     Please make sure that auto-generated configuration file looks as follows, adjust it if necessary:

     ...
     [git "ProjectA/ApplicationOne/ModuleX"]
       translationRoot = /ProjectA/ApplicationOne/ModuleX
       repository = git//ProjectA/ApplicationOne/ModuleX.git
       pathEncoding = UTF-8
       trunk = trunk:refs/heads/master
       branches = branches/*:refs/heads/*
       shelves = shelves/*:refs/shelves/*
       tags = tags/*:refs/tags/*
     ...
     

  2. Authors mapping

     At this stage you have to create /conf/authors.txt file that maps existing SVN usernames to Git authors. Please refer to documentation for more details.

  3. Installation

     Finally you have to import your Subversion repository to Git and enable synchronization by running subgit install command:

     subgit install repo
     SubGit version <VERSION> build #<BUILD_NUMBER>
     
     Subversion to Git mapping has been found:
       /ProjectA/ApplicationOne/ModuleX : <SVN_REPO>/git/ProjectA/ApplicationOne/ModuleX.git
       /ProjectA/ApplicationTwo/ModuleY : <SVN_REPO>/git/ProjectA/ApplicationTwo/ModuleY.git
       ...
     
     Processing '/ProjectA/ApplicationOne/ModuleX'
       Translating Subversion revisions to Git commits...
     Processing '/ProjectA/ApplicationTwo/ModuleY'
       Translating Subversion revisions to Git commits...
       ...
     
       Subversion revisions translated: <REVISIONS_NUMBER>.
       Total time: <TIME_SPENT> seconds.
     
     INSTALLATION SUCCESSFUL
     

  4. Git Server

     When the installation is over and synchronization between Subversion and Git repositories is enabled, you can setup some Git server (or reuse existing Apache HTTP server). Please refer to documentation on that and see a couple of posts on this topic in our blog:

     • VisualSVN Server and SubGit
     • Gitolite and SubGit

• Remote Mirror Mode

  When using this mode one has to install SubGit into Git repository only and keep this repository synchronized with remote Subversion server hosted on a different machine.

  Below you can find some basic instructions. Please refer to SubGit 'Remote Mode' Book for more details.

  1. Configuration

     In remote mirror mode SubGit does not try to auto-detect repository layout, so you have to run subgit configure --svn-url <SVN_URL> command for every module within Subversion repository:

     subgit configure --svn-url <SVN_ROOT_URL>/ProjectA/ApplicationOne/ModuleX <GIT_ROOT_DIR>/ProjectA/ApplicationOne/ModuleX.git
     SubGit version <VERSION> build #<BUILD_NUMBER>
     
     Configuring writable Git mirror of remote Subversion repository:
       Subversion repository URL : <SVN_ROOT_URL>/ProjectA/ApplicationOne/ModuleX
       Git repository location   : <GIT_ROOT_DIR>/ProjectA/ApplicationOne/ModuleX.git
     
     CONFIGURATION SUCCESSFUL
     ...
     

     As result SubGit generates configuration file <GIT_REPO>/subgit/config for every Git repository. For your case this configuration file should look as follows:

     ...
     [svn]
       url = <SVN_ROOT_URL>/ProjectA/ApplicationOne/ModuleX
       trunk = trunk:refs/heads/master
       branches = branches/*:refs/heads/*
       tags = tags/*:refs/tags/*
       shelves = shelves/*:refs/shelves/*
       fetchInterval = 60
       connectTimeout = 30
       readTimeout = 60
       keepGitCommitTime = false
       auth = default
     
     [auth "default"]
       passwords = subgit/passwd
       useDefaultSubversionConfigurationDirectory = false
       subversionConfigurationDirectory = <SVN_CONFIG_DIR>
     ...
     

  2. Authors mapping

     At this stage you have to create /subgit/authors.txt file that maps existing SVN usernames to Git authors. Please refer to documentation for more details.

  3. SVN credentials

     In case you're not using file:// protocol you have to provide necessary credentials, so SubGit is able to authenticate against Subversion server. For more information on that please read corresponding chapter in SubGit Book.

     We also recommend enabling pre-revprop-change hook on Subversion side which makes further installation and maintenance a bit easier, see SubGit Book.

  4. Installation

     Finally you have to import your Subversion repository to Git and enable synchronization by running subgit install command:

     subgit install git
     SubGit version <VERSION> build #<BUILD_NUMBER>
     
     Translating Subversion revisions to Git commits...
     
       Subversion revisions translated: <REVISIONS_NUMBER>.
       Total time: <TIME_SPENT> seconds.
     
     INSTALLATION SUCCESSFUL
     

     This command also launches background process that polls SVN server and fetches new revisions when they appear there. Basically, that means that SubGit uses dedicated process for every Git repository. Sometimes it makes sense to avoid running such processes and use some job scheduler instead.

  5. Git server

     Those links I've provided above are relevant for remote mode as well.

     However, if you're going to use Atlassian Stash for Git hosting, you can use SVN Mirror Plugin which is based on SubGit engine and provides some better experience with regards to UI and maintenance.

We have the following guideline which is based on our experience:

1. In case of many independent Subversion repositories, it's better to use SubGit in local mirror mode as it doesn't require SVN polling and maintaining additional process(es) for that.

2. In case of one giant Subversion repository with many modules, it's better to use remote mirror mode with file:// protocol and also adjust basic setup slightly.

   It definitely doesn't make sense to run 100+ background processes in your case, instead we recommend installing additional post-commit SVN hook that checks what particular modules were modified by a given revision and then triggers synchronization for corresponding Git repositories.

If you have any other questions, feel free to ask us here at Stack Overflow, at our issue tracker or contact us via email: [email protected].