Unable to use bash script to remove files from git history

Question

I have a problem with this bash script I am writing. I am trying to apply step 3 of this procedure from Github documentation to remove all files in our repositories history from our git history. For example, if .gitignore ignores .txt, I want to remove all .txt from the history. And so on for each .gitignore line.

I am able to use a wildcard in the example file. So I can run the following to remove all .txt files:

git filter-branch --force --index-filter 'git rm --cached --ignore-unmatch *.txt' --prune-empty --tag-name-filter cat -- --all

So I tried to make a bash script to do the same.

I made a simple .gitignore file to test this with, as following:

#foo
*.txt
*.docx

My shell script looks like the following:

#!/bin/bash

s1="#"
str1="'git rm --cached --ignore-unmatch "
str2="'"

cat .gitignore | while read line
do
    if [[ $line == *"$s1"* ]]
    then
        echo #skip
    else
        str3=$str1$line$str2
        git filter-branch --force --index-filter $str3 --prune-empty --tag-name-filter cat -- --all
    fi
done

However, this results in the following output:

fatal: bad revision 'rm'
fatal: bad revision 'rm'

(Once for each line of the .gitignore file)

What have I done wrong?

Answer 1

I believe this might do what you want.

#!/bin/bash

cat .gitignore | while read line
do
    if [[ $line == *#* ]]
    then
        echo #skip
    else
        git filter-branch --force --index-filter 'git rm --cached --ignore-unmatch '"$line" --prune-empty --tag-name-filter cat -- --all
    fi
done

I avoided all the pointless intermediate variables that just complicated things. (Though just removing the single quote from str1 and dropping str2 might have been enough this is saner as far as I'm concerned.)

The filter just needs to be a single string but how you construct that string is up to you.

You can probably also collect all the patterns from gitignore and just run filter-branch once (and not need --force at that point) too.

#!/bin/bash

patts=()
while read -r line
do
    if [[ $line != *#* ]]; then
        patts+=("$line")
    fi
done < .gitignore

git filter-branch --index-filter 'git rm --cached --ignore-unmatch '"${patts[*]}" --prune-empty --tag-name-filter cat -- --all

Also note that you should probably be sure that you are only dealing with simple patterns here because the rm and .gitignore understanding of patterns isn't quite the same in all cases I don't think.