Remove history for everything except a list of files using git filter-branch

Question

I'm trying to move some files between two git repositories repo1 and repo2. I have a short list of files I'd like to move (preserving history).

Three files to move from repo1:

libraryname/file1
libraryname/file2
tests/libraryname/file3

There are other files in libraryname/ and tests/libraryname/. There are other folders in / and tests/

My plan is to checkout repo1, then modify the history tree until it only contains history for the files I'm interested in. Then checkout repo2, and merge in the output of the previous operation. It seems like git filter-branch is the right tool for the first step.

So far I've tried git filter-branch --index-filter 'git rm -r --cached <FILES>' Where <FILES> lists every unwanted whole folder or file.

But this leaves a lot of folders which no longer exist at HEAD, but have existed at some point in this repositories lifetime. It seems quite tedious to figure out everything that has existed in the history of this repo - there must be a better way

How do I end up with a git commit tree which only includes these three files? Is there a better way then I'm suggesting? Or, is there a way to remove traces of all files which don't currently exist at HEAD?

Answer 1

With Git 2.24 (Q4 2019), git filter-branch is deprecated.

The equivalent would be, using newren/git-filter-repo, and its example section:

If you have a long list of files, directories, globs, or regular expressions to filter on, you can stick them in a file and use --paths-from-file; for example, with a file named stuff-i-want.txt with contents of

README.md
guides/
tools/releases
glob:*.py
regex:^.*/.*/[0-9]{4}-[0-9]{2}-[0-9]{2}.txt$
tools/==>scripts/
regex:(.*)/([^/]*)/([^/]*)\.text$==>\2/\1/\3.txt

then you could run

git filter-repo --paths-from-file stuff-i-want.txt

In your case, stuff-i-want.txt would be:

libraryname/file1
libraryname/file2
tests/libraryname/file3

---

As kubanczyk points out in the comments:

Works well on Ubuntu 20.04, you can just pip3 install git-filter-repo since it's stdlib-only and doesn't install any dependencies.

On Ubuntu 18 it's incompatible with distro's git version, but it's easy to enough to run it on a docker run -ti ubuntu:20.04