How to avoid special characters when redirecting output in bash scripts

Question

when redirecting output from a bash script to a file, I get special characters in the file. For example,

for file in *; do echo $file; done > output.txt

then if I cat output.txt

cat output.txt

I get

file1.txt
file2.txt
file3.txt
output.txt

but when editing the file, I see this:

^[[0m^[[0mfile1.txt
^[[0m^[[0mfile2.txt
^[[0m^[[0mfile3.txt
^[[0m^[[0moutput.txt

How do I avoid those nasty characters?

---

Solution:

I had the following line in the .bashrc:

trap 'echo -ne "\e[0m"' DEBUG

by removing it, I solved the problem.

Thank you all for your help.

Answer 1

These are ANSI escape codes, used for formatting text in a terminal. Rather than trying to remove them, you should prevent them from being written in the first place.

Are you sure you're getting this from the exact code you posted? If so, your files actually have these characters in your names, and you should simply rename them.

The far more common way of seeing this is having tools that output ANSI escapes sequences. This is a reproducible way of showing the same issue:

ls --color=always > file

If your posted code was an untested example, you should go through and find the tool responsible for the ANSI codes and make it stop (make especially sure you're not looping over ls output).

Here's an example of the problem you're seeing, with touch as stand-in for some process/script that accidentally created filenames with ANSI escapes:

# Reproduce the problem
$ touch $'\x1B[0m\x1B[0mfile.txt'

# Symptoms of the problem
$ ls *.txt
?[0m?[0mfile.txt

$ for f in *.txt; do echo "$f"; done
file.txt

$ for f in *.txt; do echo "$f"; done | cat -v
^[[0m^[[0mfile.txt

# Fix the problem by renaming the bad files
$ crud=$'\x1B[0m'; for f in *"$crud"*; do mv "$f" "${f//$crud/}"; done

# Now works as expected
$ ls *.txt
file.txt

$ for f in *.txt; do echo "$f"; done
file.txt

$ for f in *.txt; do echo "$f"; done | cat -v
file.txt

Answer 2

Run the raw ls command

/bin/ls

as

/bin/ls > your_file

and you will avoid the special characters in the output file your_file.