Splitting huge git repo

Question

I am trying to split a git repo using the recipe described in this question, but the git repo in question is huge and has thousands of commits, so command line limit kicks in when I try to run the command as described and the commit filter fails. If I try to do it in stages (first applying index filter, then commit filter), the index filter works fine, but the commit filter gets stuck and starts to use huge amounts of memory (3G+) so I have to kill it. Any idea how to do this? Maybe I'm doing something wrong here?

For those not reading the other question, the task is to split couple of directories from a very big git repo into another repo, while preserving history that pertains to these directories.

Answer 1

I think that this has been added since I wrote that answer:

--prune-empty

Some kind of filters will generate empty commits, that left the tree untouched. This switch allow git-filter-branch to ignore such commits. Though, this switch only applies for commits that have one and only one parent, it will hence keep merges points. Also, this option is not compatible with the use of --commit-filter. Though you just need to use the function git_commit_non_empty_tree "$@" instead of the git commit-tree "$@" idiom in your commit filter to make that happen.

So give it a shot with git filter-branch --index-filter '...' --prune-empty, since it's just the commit filter in my other answer that was giving you problems.