Stop git from silently clobbering ignorefiles (bug in git?)?

Question

I have found a way that git will not ask you to stash but instead silently delete your files that you believed to be safe in .gitignore. This is true even if you had the same ignore file since your initial commit.

The problem occurs when you are in a commit that has a said file removed but listed in .gitignore and then you checkout another commit where it existed:

git init
mkdir ignored/
echo stuff > ignored/file
echo otherstuff > otherfile
git add -A
# Opps added my ignored folders files
# Forgeting to rm --cache
echo ignored/ > .gitignore ; git add .gitignore
git commit -m 'accidentally add ignored/file'
git status
touch dummyfile; git add dummyfile
# Remembered to rm --cache
git rm --cache -rf ignored/file #This file is now vulnerable to clobber
git commit -m 'add stuff'
echo somechange >> ignored/file

## Wait for it..
git checkout HEAD~
## somechange has been silently clobbered!!

# Please paste first paragraph, observe and then past the second.
# Note both commits have the correct ignore file and are not immune!

(cd to an empty folder before pasting above code into terminal)

Is there anyway to prevent this silent clobber?

Answer 1

It's not a bug (or at least, the git developers don't consider it to be one).

The contents of .gitignore are not "files that should be ignored" or "path names that should not be touched"; instead, they're "paths that don't get automatically added, and are suppressed from being shown as untracked" (which makes .gitignore a poor name).

Once you've committed a file, or even added it to the index, it's no longer ignored, regardless of whether it is listed in a .gitignore.

For some files, you can use git update-index --assume-unchanged or (better) git update-index --skip-worktree, but in general, if you've accidentally committed a file you should not have, and you want it ignored, you must "rewrite history" to take it entirely out of the repository to get good behavior. That's not too difficult if you haven't pushed anything and have few commits that contain the unwanted file, but much harder if you have pushed, or have many such commits.

See also Git - Difference Between 'assume-unchanged' and 'skip-worktree'.

[Text below added December 2016]

Technical details

For Git, there is a short, simple, and sweet definition of a tracked file: A file is tracked if and only if there is an entry for it in the index.

The "assume unchanged" and "skip worktree" things are flags you can manually set or clear in the index. They are separate flags and can be set individually, although it's not clear what it means to set both. The intent of "assume unchanged" is merely to make Git faster, by allowing it to assume that the file is not changed and hence does not need to be updated by git add, while the intent of the "skip worktree" flag is to tell Git "hands off": don't just assume it's unchanged, try hard to keep it unchanged. That's harder than it may look at first blush; for more about this, see the above linked post.

In order to set these flags, the file must have an index entry, so it must be tracked.

If a file is not tracked, it may or may not also be ignored. There are numerous sources for these: not just the top-level .gitignore, which contains a list (one per line) of a variant of Git's pathspec that is limited to glob matching and negation, but also .gitignore files within each sub-directory within a repository, the file $GIT_DIR/info/exclude if it exists, and the file named by core.excludesFile if that configuration entry exists. To find whether and why a file is ignored, use git check-ignore, available since Git version 1.8.2. The -v option tells you which control file marked the file as ignored.

Unfortunately, as in the question to which this is an answer, "ignored" has two meanings. While matching an ignore path keeps Git quiet about the file being untracked, it also makes Git feel free to clobber the file.