grep -f on OS X produces segfault

Question

If you've got a Mac, try this:

echo 'abcd*' > grepfile
echo 'abc$' >> grepfile
echo '^abc' >> grepfile
echo "fojeiwuroiuwet\nljfajsljkfabcdddjlfkajlkj\nabcaaa\nzzzabc\n" | grep -f grepfile

Here's the version:

$ grep --v
grep (BSD grep) 2.5.1-FreeBSD

This is a machine (a rMBP of the 2012 flavor) that's kept up with Apple's software updates, so I am on 10.8.4.

I verified that GNU grep compiled from source does not suffer from this problem. Indeed it is version 2.14, which is a whole lot of versions past 2.5.1.

But how might one achieve the task of testing some input against a series of regexes otherwise, without some vastly inefficient loop that forks a grep for each regex?

Edit: The approach I took to work around this was something akin to: while read REGEX; do [[ ... =~ $REGEX ]] ... done < regexfile.

Question: Is this a known bug with this version of grep? How can we set up our systems so they will work properly with a file of regexes to grep?

Update: Looks like some folks are reporting it works fine (even with this particular FreeBSD 2.5.1 grep). What are some steps I can take to try to figure out which .so/.dylib's it might be using? Can someone do a shasum /usr/bin/grep and tell me if it works for you? (I'm not sure if that would provide much information, but what I'm after is whether my computer's configuration is screwed up, or if this is some actual existing issue with this version of the software.)

$ shasum /usr/bin/grep
eac59389d09642decbb8551e2c975f795934bfbf  /usr/bin/grep

Here is more info:

$ codesign -dvvv /usr/bin/grep
Executable=/usr/bin/grep
Identifier=com.apple.zgrep
Format=Mach-O thin (x86_64)
CodeDirectory v=20100 size=224 flags=0x0(none) hashes=6+2 location=embedded
Hash type=sha1 size=20
CDHash=93b823c000188f8737653d8333c90a6db9361d70
Signature size=4064
Authority=Software Signing
Authority=Apple Code Signing Certification Authority
Authority=Apple Root CA
Info.plist=not bound
Sealed Resources=none
Internal requirements count=2 size=208

Further investigation:

$ gdb /usr/bin/grep
GNU gdb 6.3.50-20050815 (Apple version gdb-1824) (Thu Nov 15 10:42:43 UTC 2012)
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "x86_64-apple-darwin"...Reading symbols for shared libraries .... done

(gdb) start -f grepfile
Function "main" not defined.
Make breakpoint pending on future shared library load? (y or [n])
Starting program: /usr/bin/grep -f grepfile
Reading symbols for shared libraries +++.............................. done
abc
abc

Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: KERN_INVALID_ADDRESS at address: 0x0000000101000000
0x00007fff89b5d1b3 in memchr ()
(gdb) where
#0  0x00007fff89b5d1b3 in memchr ()
#1  0x00007fff89b8e45a in __sfvwrite ()
#2  0x00007fff89b8e861 in fwrite ()
#3  0x0000000100003138 in _mh_execute_header ()
#4  0x0000000100002988 in _mh_execute_header ()
#5  0x0000000100001c28 in _mh_execute_header ()
#6  0x00007fff8e2d57e1 in start ()
(gdb)

I have rebooted the machine as well. It repeatably does the same thing in gdb.

Answer 1

I've got OSX 10.8.4 on MacBook Air and your example doesn't crash by default, but only when adding --color parameter.

Explanation

This crash usually happens when you're mixing wilcard (asterisk sign) with the terminal colours and this is the software bug.

Also check another simpler example:

echo 'abc*' | grep --color=auto -e ".*" -e a

Here it seems that --color=auto makes the difference (without it or setting to never, then it doesn't crash). So I assume that your grep is using colors in terminal by default.

Solution

Make sure that your grep is not an alias to the grep with colors enabled, or the colors are not enabled by default.

You can always try to run grep with --color=never.

For permanent solution, I've reported the bug report:

http://www.freebsd.org/cgi/query-pr.cgi?pr=181263

so the issue can be fixed in the software it-self.

---

If you'd like to access more detailed log of your crash, go to Console, Show Log List and find the crash log under User Diagnostic Reports.

E.g.:

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   libsystem_c.dylib               0x00007fff8a8da1b3 memchr + 15
1   libsystem_c.dylib               0x00007fff8a90b45a __sfvwrite + 578
2   libsystem_c.dylib               0x00007fff8a90b861 fwrite + 114
3   grep                            0x000000010a4a3138 0x10a4a0000 + 12600
4   grep                            0x000000010a4a2988 0x10a4a0000 + 10632
5   grep                            0x000000010a4a1c28 0x10a4a0000 + 7208
6   libdyld.dylib                   0x00007fff8daf77e1 start + 1

---

Here is some more detailed explanation about the crash:

A quick test

  $ env -i bsdgrep -Fi without_nls usr.bin/grep/grep.c
  $ env -i gnugrep -Fi without_nls usr.bin/grep/grep.c
  #ifndef WITHOUT_NLS
  #ifndef WITHOUT_NLS
  #ifndef WITHOUT_NLS

shows that bsd fgrep already fails to ignore case. And if you throw
a few more options to the mix it'd crash, e.g.

  $ env -i LC_CTYPE=en_US.UTF-8 TERM=xterm bsdgrep --color -Fir without_nls usr.bin/grep/
  [...]
  Program received signal SIGSEGV, Segmentation fault.
  0x0000000801007ff2 in memchr (s=0x61167a, c=10, n=18446744073707490297) at /usr/src/lib/libc/string/memchr.c:48
  48                              if (*p++ == (unsigned char)c)
  (gdb) bt
  #0  0x0000000801007ff2 in memchr (s=0x61167a, c=10, n=18446744073707490297) at /usr/src/lib/libc/string/memchr.c:48
  #1  0x0000000801007b03 in __sfvwrite (fp=0x801247770, uio=0x7fffffffd8f0) at /usr/src/lib/libc/stdio/fvwrite.c:170
  #2  0x0000000801007698 in fwrite (buf=0x608c03, size=18446744073709551606, count=1, fp=0x801247770)
      at /usr/src/lib/libc/stdio/fwrite.c:95
  #3  0x0000000000405498 in printline (line=0x7fffffffdb70, sep=58, matches=0x7fffffffd990, m=9)
      at /usr/src/usr.bin/grep/util.c:500
  #4  0x0000000000404f51 in procline (l=0x7fffffffdb70, nottext=0) at /usr/src/usr.bin/grep/util.c:381
  #5  0x000000000040489f in procfile (fn=0x80140b600 "usr.bin/grep/nls/es_ES.ISO8859-1.msg") at /usr/src/usr.bin/grep/util.c:239
  #6  0x00000000004044d7 in grep_tree (argv=0x7fffffffdd30) at /usr/src/usr.bin/grep/util.c:163
  #7  0x0000000000403ea9 in main (argc=5, argv=0x7fffffffdd10) at /usr/src/usr.bin/grep/grep.c:689

Source: http://lists.freebsd.org/pipermail/freebsd-current/2011-August/026502.html

---

Also it seems that there are different grep binaries on different OSX even with the same version:

$ grep --v
grep (BSD grep) 2.5.1-FreeBSD
$ shasum /usr/bin/grep
350ee11e1868e18c9707ea7035184a114f40badf  /usr/bin/grep
$ codesign -dvvv /usr/bin/grep
Executable=/usr/bin/grep
Identifier=com.apple.zgrep
Format=Mach-O thin (x86_64)
CodeDirectory v=20100 size=224 flags=0x0(none) hashes=6+2 location=embedded
Hash type=sha1 size=20
CDHash=1537b3ed49878d5d18482859a37318164a2a40f1
Signature size=4064
Authority=Software Signing
Authority=Apple Code Signing Certification Authority
Authority=Apple Root CA
Info.plist=not bound
Sealed Resources=none
Internal requirements count=2 size=176