Why does Git subtree push take longer the more commits I have?

Question

I've been using the git subtree extension (https://github.com/apenwarr/git-subtree) . I use "--squash" to make the log of main project clean, my steps like this:

1. add lib into main project

   git subtree add -P sub/libdir --squash lib_remote master

2. get update from lib

   git subtree pull -P sub/libdir --squash lib_remote master

3. push changes to lib_remote

   git subtree push -P sub/libdir --squash lib_remote master

It works very well for me(both main project and lib, have a history make good sense). The problem is the time of git subtree push, becomes longer and longer.

My purpose of using git-subtree is almost the same with Screndib, who asked git-subtree is not retaining history so I cannot push subtree changes, how can I fix this/avoid this issue in the future?

I guess, when using --squash, every time to process a push , git subtree needs to search the whole history since the "subtree add".

How can I reduce the time of subtree push? Or make it work more effective, rather than the whole history, only process changes since last git subtree push(or pull)?

Answer 1

I assume "lib_remote" in your code menas the url of the remote lib repo not a branch in your current repo? Both remote repo url and branch in your current repo are working.

I see you were using git subtree add to add the remote lib as subtree and then you just using git subtree push to push the changes.

It is better to do a git subtree split operation to split the subtree changes to a seperate branch in your curent repo before the push operation, then push the splited branch to the remote repo and keep this splited branch existed, every time before the pushing, do a git subtree split operation again, this will build the history of the subtree form the point you last splited, it will be much faster. Otherwise, without this split just like you did, git subtreee has to build the history of the subree from the point you added, as long as the subtree commits growing,the time of the building will get longer and longer.

If you are not use --squash while adding, you can consider using --rejoin while you spliting, this will be much faster.

So, the step should be the following.

1. add lib into main project

   git subtree add -P sub/libdir --squash lib_remote_url master

2. get update from lib

   git subtree pull -P sub/libdir --squash lib_remote_url master

3. split the subtree changes to a seperate branch

   git subtree split -P sub/libdir -b lib_remote_branch

4. push changes to lib_remote

   git push lib_remote_url lib_remote_branch:master

Keep the lib_remote_branch existed and redo the step 3 and step 4 when next time push.

Answer 2

The main problem is with the new commits on different prefixes since you did your first add. My solution to this problem is to do a "clean-up" operation where you filter all of your changes that are not in other repositories and then do a git subtree add on this new branch.

This is a rather brute-force way of doing it but it's basically the same as saying do a "git subtree add" on a fresh copy of your main repository. That's the only thing that worked for me...