Problems with git reset --hard after adding .gitattribute -- local changes that should not be there

Question

I have observed CRLF problems after merging bundles from my colleague. Sometimes lines with LF were mixed into the sources, probably the ones that were merged into. Therefore, we have decided to add the .gitattributes file with the following content (comments removed):

*.cpp text
*.h text
*.inc text
*.cfg text
*.dic text

*.sln text eol=crlf
*.vcxproj text eol=crlf
*.filters text eol=crlf
*.user text eol=crlf
*.rc text eol=crlf
*.rc2 text eol=crlf

Now I observe the strange behaviour. I can see a lot of modified: ... files (i.e. unstaged) that should not be there. I tried git reset --hard, but the files still have the same status. I tried to clone the repository again -- the same result.

I have git version 1.7.11.msysgit.0 installed from the Git-1.7.11-preview20120620.exe downloaded as current version for Windows.

What else should I try?

Thanks, Petr

Answer 1

The Cause

It is * text=auto setting in .gitattributes that causes this issue. You can remove it and live happily ever after, but you may have files with non-repo-default line encodings, or even files with several different line ending encoding (i.e. both LF and CRLF, and even CR!) in your repo.

Why This Happens (Details)

When git checks out a file as-is, on which it will modify line endings upon add/commit. The file is in fact not modified yet, but git already considers it as modified, because it will be, due to the settings of the repo.

Somehow it works a little bit weird with git. For example, git reset --hard sometimes works and sometimes not, perhaps depending on your settings. Or, if you go into your .gitattributes and mark the extension as binary, the modified file magically disappears:

*.ext binary

The effect stays even after you remove the binary marking, even after you do git reset --hard again, so it could be a git bug or git caching issue. Doing git -rm on the file and then doing git reset --hard restores the modified marking.

How to fix it

We assume here that you want to keep your * text=auto setting, so that git warns you of inconsistent line endings in various textual files now, and in the future. If so, pick your method:

Option 0: Temporarily fool git from marking files as modified

1. Edit .gitattributes, comment out * text=auto, save
2. git status (this step is needed to have git record changes in .gitattributes)
3. git reset --hard (this will restore * text=auto and also nuke any changes in your working directory, if you've made any).

This usually works (except in perhaps most stubbornest of cases). It also postpones the problem, which most likely will show up later at some point, because line endings still have not been normalized.

This option is great when you have to rewind a previous commit that has not been normalized, like during a rebase, or some other git work, where you know existing following commits will normalize the line endings, but git complains about modified files now preventing you from continuing. So basically use this method when you need git to shut up and ignore modified files that really have not been modified for your particular context.

Option 1: Easy "end-user" fix

If you have just a few files, ensure your .gitattributes and core.autocrlf are set to your liking, and then just make a git add/commit and you should not see this issue again. The files will be converted to your desired line endings and stored in your repo, as stated in your configuration. This commit will be stored in your repo as "entire file had changed", because every line will have its line endings changed. For a few files in a larger or open-source repo that is fine. Be sure to merge or cherry-pick that commit into all of your branches, as the issue will exist in all branches that had those files, until you fix it. By the way here is where you may use Option 0. i.e. if you switch to an unfixed branch, and it complains, run Option 0, then do the fix (merge or cherry-pick).

IMPORTANT: if you are going this route of Option 1, be sure to convert the modified files properly. git may not be doing it for you quite as you expect, so do it yourself prior to your commit, i.e. use this: Converting newline formatting from Mac to Windows The reason git may be confused is I have seen files that have all three CR, LF, and CRLF line ending formatting in them. Nuke those into your preferred format yourself before commit.

Option 2: Advanced mechanics "git history rewriting" fix:

If you have a more private repo and are not afraid to rewrite history, go see this: git sees entire file as one line due to mac line endings This will rewrite entire repo and get rid of any line ending issues anywhere on all trees, branches, forever! Be sure to include all potential troublesome text-file extensions that you may want to normalize, or else they may show up later.

In my case I did Option 2 as I was dealing with in lots of file ending issues in lots of branches. But then I had some unexpected extensions show up that I did not normalize, and just did Option 1, as I only missed 5-6 files.