git submodule commit/push/pull

Question

I'd like to use git submodule.

The steps I need to take to push my changes to my project are

1. add/commit/push from submodule directory
2. add/commit/push from parent directory

Steps I need to take to pull changes of my project.

1. git pull from parent directory
2. git submodule update from parent directory

Steps for updating the submodule from its original repo

1. git pull from submodule directory

What worries me is the following exerpt from http://git-scm.com/book/en/Git-Tools-Submodules

The issue is that you generally don’t want to work in a detached HEAD environment, because it’s easy to lose changes. If you do an initial submodule update, commit in that submodule directory without creating a branch to work in, and then run git submodule update again from the superproject without committing in the meantime,(?? update/commit/update will lose change?) Git will overwrite your changes without telling you. Technically you won’t lose the work, but you won’t have a branch pointing to it, so it will be somewhat difficult to retrieve.

To avoid this issue, create a branch when you work in a submodule directory with git checkout -b work or something equivalent. When you do the submodule update a second time, it will still revert your work, but at least you have a pointer to get back to.

I'm going to modify submodules and don't wanna mess up, the doc above briefly mentions the possibility of losing change, and I don't understand what might cause the loss.

I wonder what additional steps more than I listed above I need to take to prevent the loss. Especially several team members modify submodules, what do they need to do not to mess up?

Answer 1

I'd like to share my experiences with you as someone who works with external projects in Visual Studio solutions trying to solve a similar problem. I'm relatively new to git, so if anyone has any constructive criticism I would appreciate that.

If you're using Visual Studio, the Git Source Control Provider extension is free (http://visualstudiogallery.msdn.microsoft.com/63a7e40d-4d71-4fbb-a23b-d262124b8f4c), and seemed to recursively commit submodules when I tested it out.

HOWEVER I'm using VS Web Developer Express at home for development, so I don't want to rely on an extension (I also think it's good to have some idea of what's happening under the hood). Therefore I've been forced to figure out the commands, and I've added some notes below.

NOTES

If you haven't done already, have a thorough read through http://git-scm.com/book/en/Git-Tools-Submodules. There are quite a few caveats, and I will refer back to this page. If you try to work with submodules without reading this, you will give yourself a headache very quickly.

My approach follows this tutorial, with a few added extras: http://blog.endpoint.com/2010/04/git-submodule-workflow.html

Once you have your superproject initialised (e.g. git init && git remote add origin ...), start adding your submodules like so:

git submodule add git://github.com/you/extension1.git extension
git submodule init
git submodule update

Check that your .gitmodules file reflects this addition, e.g.

[submodule "extension1"]
        path = extension
        url = git://github.com/you/extension1.git

Switch to your submodule directory (i.e. cd extension). Run:

git fetch #I use fetch here - maybe you can use pull?
git checkout -b somebranchname #See the Git-Tools-Submodules link above for an explanation of why you need to branch

I made a change here to README.txt so I could commit it (also so I would have a record of what I was doing in this commit), then commited the module to apply the branch (still within the submodule directory):

git add .
git commit -a -m "Branching for extension submodule"

Now go into the superproject (i.e. cd ..). You will also need to commit here (if you look at the git submodule page I mentioned it explains why this is necessary):

git status #will show you that your submodule has been modified
git commit -a -m "Commiting submodule changes from superproject"

Now we can recusively push our projects like so if required:

git push --recurse-submodules=on-demand

You will need to run through the above steps once for all your submodules.

Once you have done this for all your submodules and started making changes you want to commit and push, you can use:

git submodule foreach 'git add .' #recursively add files in submodules

Unfortunaly I haven't found a way to recursively commit without using something like git-slave (anyone?), so you then need to go into each submodule directory and run a regular commit for the files you just added. In the superproject:

git status #tells you that `extension` submodule has been modified
cd extension
git commit -a -m "Commiting extension changes in superproject edit session"

Once the submodule is commiting, you'll also need to commit the superproject (again), so:

cd ..
git add .
git commit -a -m "Altered extension submodule"
git status #should now show 'working directory clean', otherwise commit other submodules in the same manner

This can get slightly annoying (because you end up committing twice), but once you're aware of it it's actually not so bad (as it forces you to check what you're committing in each project). Just my opinion - if you've isolated some of your superproject's functionality into submodules, it should work in isolation from the rest of your projects anyway (so commiting them at different times while annoying is not the end of the world).

Now we can push once more...

git push --recurse-submodules=on-demand

If you then descend into your submodule and try and push again, you'll find it won't do anything as the latest commit has already been pushed.

Cloning (or using a remote origin) for a superproject can also be quite confusing - such as the need to run git submodule update twice after git submodule init. Read the 'Cloning a Project with Submodules' section of http://git-scm.com/book/en/Git-Tools-Submodules.

Something that caught me out when cloning my superproject was getting the latest changes for the submodules. See Easy way pull latest of all submodules

My variant of this is to use a 'development' branch for checked out submodules (but you can call it whatever you want) and then use this in the superproject:

git submodule foreach git pull origin development

When I set this up I also swap to the branch I want to push my changes to on the checked out submodule like so:

cd extension
git checkout -b development #This will tell you this is a new branch, but I believe this means a new branch of the local git repository - this will get pushed to the 'development' branch
#Make your changes, commit etc.

I can confirm that when I follow the above steps, changes to the submodules in a clone/remote origin project (when pushed) showed up in other clones/remote origins of the same project (not forgetting that last submodule pull command).

I hope that was of some use to you.