Strange behaviour of Git: mysterious changes cannot be undone

Question

I am seeing a behaviour in Git which seems very mysterious to me. I keep a clone of the Linux repository from Github to play with Git locally. To be clear, I don't do much in this repository : I fetch changes, update master, checkout a specific version, and sometimes I try out a Git GUI to see what the visualization looks like on a big project. TLDR version: I never did any changes to the files in it.

Strange behaviour

Earlier today, I checked-out master and pulled changes from Github. Everything seemed to go fine. But I suspect it actually didn't. This is what git status looks like now.

axel@macbook ~/Depots/linux $ git status
# On branch master
# Changes not staged for commit:
#   (use "git add <file>..." to update what will be committed)
#   (use "git checkout -- <file>..." to discard changes in working directory)
#
#   modified:   include/linux/netfilter/xt_connmark.h
#   modified:   include/linux/netfilter/xt_dscp.h
#   modified:   include/linux/netfilter/xt_mark.h
#   modified:   include/linux/netfilter/xt_rateest.h
#   modified:   include/linux/netfilter/xt_tcpmss.h
#   modified:   include/linux/netfilter_ipv4/ipt_ecn.h
#   modified:   include/linux/netfilter_ipv4/ipt_ttl.h
#   modified:   include/linux/netfilter_ipv6/ip6t_hl.h
#   modified:   net/ipv4/netfilter/ipt_ecn.c
#   modified:   net/netfilter/xt_dscp.c
#   modified:   net/netfilter/xt_hl.c
#   modified:   net/netfilter/xt_rateest.c
#   modified:   net/netfilter/xt_tcpmss.c
#
# Untracked files:
#   (use "git add <file>..." to include in what will be committed)
#
#   samples/hidraw/
no changes added to commit (use "git add" and/or "git commit -a")

Where do these changes come from, I wonder. Now comes the worst part. Let us see what happens if I try to get rid of those files and check them out again.

axel@macbook ~/Depots/linux $ rm -Rf include net
axel@macbook ~/Depots/linux $ git checkout -- .
axel@macbook ~/Depots/linux $ git status
# On branch master
# Changes not staged for commit:
#   (use "git add <file>..." to update what will be committed)
#   (use "git checkout -- <file>..." to discard changes in working directory)
#
#   modified:   include/linux/netfilter/xt_CONNMARK.h
#   modified:   include/linux/netfilter/xt_DSCP.h
#   modified:   include/linux/netfilter/xt_MARK.h
#   modified:   include/linux/netfilter/xt_RATEEST.h
#   modified:   include/linux/netfilter/xt_TCPMSS.h
#   modified:   include/linux/netfilter_ipv4/ipt_ECN.h
#   modified:   include/linux/netfilter_ipv4/ipt_TTL.h
#   modified:   include/linux/netfilter_ipv6/ip6t_HL.h
#   modified:   net/ipv4/netfilter/ipt_ECN.c
#   modified:   net/netfilter/xt_DSCP.c
#   modified:   net/netfilter/xt_HL.c
#   modified:   net/netfilter/xt_RATEEST.c
#   modified:   net/netfilter/xt_TCPMSS.c
#
# Untracked files:
#   (use "git add <file>..." to include in what will be committed)
#
#   samples/hidraw/
no changes added to commit (use "git add" and/or "git commit -a")

Well, same thing. The git checkout operation seems to produce files with mysterious changes. I tried to investigate further, and I think I eliminated the possibility that these changes were caused by line-ending issues. See the beginning of a git diff below

diff --git a/include/linux/netfilter/xt_CONNMARK.h b/include/linux/netfilter/xt_CONNMARK.h
index 2f2e48e..efc17a8 100644
--- a/include/linux/netfilter/xt_CONNMARK.h
+++ b/include/linux/netfilter/xt_CONNMARK.h
@@ -1,6 +1,31 @@
-#ifndef _XT_CONNMARK_H_target
-#define _XT_CONNMARK_H_target
+#ifndef _XT_CONNMARK_H
+#define _XT_CONNMARK_H

-#include <linux/netfilter/xt_connmark.h>
+#include <linux/types.h>

-#endif /*_XT_CONNMARK_H_target*/
+/* Copyright (C) 2002,2004 MARA Systems AB <http://www.marasystems.com>
+ * by Henrik Nordstrom <[email protected]>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ */

If I understand this right, this shows changes that cannot have been made by just changing some line endings, right? Last thing I did was: try to find out the originator of the changes, but obviously that did not work. See the following git blame output.

axel@macbook ~/Depots/linux $ git blame include/linux/netfilter/xt_CONNMARK.h
00000000 (Not Committed Yet 2011-10-25 20:00:56 +0200  1) #ifndef _XT_CONNMARK_H
00000000 (Not Committed Yet 2011-10-25 20:00:56 +0200  2) #define _XT_CONNMARK_H
2e4e6a17 (Harald Welte      2006-01-12 13:30:04 -0800  3) 
00000000 (Not Committed Yet 2011-10-25 20:00:56 +0200  4) #include <linux/types.h>
0dc8c760 (Jan Engelhardt    2008-01-14 23:38:34 -0800  5) 
00000000 (Not Committed Yet 2011-10-25 20:00:56 +0200  6) /* Copyright (C) 2002,2004 MARA Systems AB <http://www.marasystems.com>
00000000 (Not Committed Yet 2011-10-25 20:00:56 +0200  7)  * by Henrik Nordstrom <[email protected]>
00000000 (Not Committed Yet 2011-10-25 20:00:56 +0200  8)  *
00000000 (Not Committed Yet 2011-10-25 20:00:56 +0200  9)  * This program is free software; you can redistribute it and/or modify
00000000 (Not Committed Yet 2011-10-25 20:00:56 +0200 10)  * it under the terms of the GNU General Public License as published by
00000000 (Not Committed Yet 2011-10-25 20:00:56 +0200 11)  * the Free Software Foundation; either version 2 of the License, or
00000000 (Not Committed Yet 2011-10-25 20:00:56 +0200 12)  * (at your option) any later version.
00000000 (Not Committed Yet 2011-10-25 20:00:56 +0200 13)  */

Questions

What am I missing? When could I have gone wrong, and how to fix this? Thanks for your tips and remarks!

Answer 1

The Linux source tree has filenames which differ in case only, which causes interesting failures on systems with case-insensitive filesystems.

You need a case-sensitive filesystem in order to work with the Linux source.

(include/linux/netfilter/xt_connmark.h and include/linux/netfilter/xt_CONNMARK.h are two different files in the Git repository, but only one can exist in your checkout at a time if your filesystem is case-insensitive.)

Answer 2

As @ephemient said, this is because case insensitive file system. And I guess you are using Mac's HFS?

For a simple solution on Mac, you can create a disk image, format the disk image with 'Case-sensitive Journaled HFS+':

hdiutil create -type SPARSE -fs 'Case-sensitive Journaled HFS+' -size 40g ~/git.dmg

Then mount the disk image by open ~/git.dmg.

And then do all git clone, git checkout operations in the mountable volumes.

---

Or you need some third-party tool to convert HFS+ from case insensitive to case sensitive. As I know, some Mac Apps are made on case insensitive file systems, so some of them may failed to work if you do this convertion.