Why am I getting some extra, weird characters when making a file from grep output?

Question

I am doing a very basic command that never gave me trouble in the past, but is inexplicably returning undesired characters now.

I am in BASH on linux, and simply want to search through a directory and make a file containing filenames that match a pattern:

ls | grep "*.file_ID" > my_list.txt

...This works fine, and if I cat the data:

   cat my_list.txt
       seriesA.file_ID
       seriesB.file_ID
       seriesC.file_ID

However, when I try to feed this file into downstream processes, I keep getting a weird errors, as if the file isn't properly formatted as a list of file names. When I open the file in vim to reveal any unnecessary characters, I find the file actually looks like this:

vi my_list.txt

^[[00mseriesA.file_ID^[[00m
^[[00mseriesB.file_ID^[[00m
^[[00mseriesC.file_ID^[[00m

For some reason, every line is started and ended with the characters ^[[00m. If I delete these characters, all of the downstream processes work fine. However, I need to have my scripts automatically make such a file list, so I can't keep going in and manually deleting these chars.

Does anyone know what is producing the ^[[00m characters? I don't have any idea where they are coming from, and need a to be able to generate files without them.

Thanks!

Answer 1

Probably your GREP_OPTIONS environment variable contains --color=always, which causes the output to be stuffed with control characters, even when piped to a file.

Use --color=auto instead.

http://www.gnu.org/software/grep/manual/html_node/Environment-Variables.html

Even better, don't use grep:

ls *.file_ID > my_list.txt

Answer 2

• usually it is automatically => grep --color=auto

• if it pipes into a csv file that looks like this ^[[00...^[[00m

• you would have to type this in the terminal:
  grep --color=auto "your regex" > example.csv

• if you want it to be a permanent situation where you do not have to type "--color=auto" every time, type this in the terminal:
  export GREP_OPTIONS='--color=auto'

more info: https://linuxcommando.blogspot.com/2007/10/grep-with-color-output.html