Git filter-branch with index-filter does not work and remove directories as expected

Question

Structure of Git repo foo in master branch

foo/refs/a.txt  
foo/bar/refs/b.txt  

In other branches refs/ might be in lots of other places

Goal

To remove all instances of the directory refs (and their content) from Git (history) Environment: Windows 7 using Git Bash

Removing refs (Git not involved, tried this just to see that it works by itself)

find . -name refs -depth -exec rm -rf {} \;

Success, all refs/ and their content are removed (If I don't use -depth, find will report an error that the dirs don't exists even though they were removed correctly).

Removing refs from Git

git filter-branch --index-filter \
'find . -name refs -depth -exec git rm -rf --cached --ignore-unmatch {} \;' \
--prune-empty --tag-name-filter cat -- --all

As can be seen in the picture (think of temp/a as temp/foo) the command runs through and rewrites all commits but no refs/ are removed so somehow the output of the find is not returned to filter-branch --index-filter as expected.

Similar things seem to work for others.
What am I missing?

PS. Yes I've read hundreds of posts, articles etc for hours and hours about this but it doesn't work for me anyway.