Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Escape dollar sign in regexp for sed

I will introduce what my question is about before actually asking - feel free to skip this section!

Some background info about my setup

To update files manually in a software system, I am creating a bash script to remove all files that are not present in the new version, using diff:

for i in $(diff -r old new 2>/dev/null | grep "Only in old" | cut -d "/" -f 3- | sed "s/: /\//g"); do echo "rm -f $i" >> REMOVEOLDFILES.sh; done

This works fine. However, apparently my files often have a dollar sign ($) in the filename, this is due to some permutations of the GWT framework. Here is one example line from the above created bash script:

rm -f var/lib/tomcat7/webapps/ROOT/WEB-INF/classes/ExampleFile$3$1$1$1$2$1$1.class

Executing this script would not remove the wanted files, because bash reads these as argument variables. Hence I have to escape the dollar signs with "\$".

My actual question

I now want to add a sed-Command in the aforementioned pipeline, replacing this dollar sign. As a matter of fact, sed also reads the dollar sign as special character for regular expressions, so obviously I have to escape it as well. But somehow this doesn't work and I could not find an explanation after googling a lot.

Here are some variations I have tried:

echo "Bla$bla" | sed "s/\$/2/g"        # Output: Bla2
echo "Bla$bla" | sed 's/$$/2/g'        # Output: Bla
echo "Bla$bla" | sed 's/\\$/2/g'       # Output: Bla
echo "Bla$bla" | sed 's/@"\$"/2/g'     # Output: Bla
echo "Bla$bla" | sed 's/\\\$/2/g'      # Output: Bla

The desired output in this example should be "Bla2bla". What am I missing? I am using GNU sed 4.2.2

EDIT

I just realized, that the above example is wrong to begin with - the echo command already interprets the $ as a variable and the following sed doesn't get it anyway... Here a proper example:

  1. Create a textfile test with the content bla$bla
  2. cat test gives bla$bla
  3. cat test | sed "s/$/2/g" gives bla$bla2
  4. cat test | sed "s/\$/2/g" gives bla$bla2
  5. cat test | sed "s/\\$/2/g" gives bla2bla

Hence, the last version is the answer. Remember: when testing, first make sure your test is correct, before you question the test object........

like image 740
Captain Ahab Avatar asked Feb 19 '16 13:02

Captain Ahab


People also ask

What does dollar sign mean in sed?

In Unix, environment variables begin with a dollar sign, such as $TERM, $PATH, $var or $i. In sed, the dollar sign is used to indicate the last line of the input file, the end of a line (in the LHS), or a literal symbol (in the RHS).

Can you use regex in sed?

Regular expressions are used by several different Unix commands, including ed, sed, awk, grep, and to a more limited extent, vi.

What does dollar do in regex?

If a dollar sign ( $ ) is at the end of the entire regular expression, it matches the end of a line. If an entire regular expression is enclosed by a caret and dollar sign ( ^like this$ ), it matches an entire line. So, to match all strings containing just one characters, use " ^.

How do I escape the dollar sign in bash?

Escaping Special Characters To tell the shell to ignore the special meaning normally associated with the dollar sign, put a backslash in front of it.


1 Answers

The correct way to escape a dollar sign in regular expressions for sed is double-backslash. Then, for creating the escaped version in the output, we need some additional slashes:

cat filenames.txt | sed "s/\\$/\\\\$/g" > escaped-filenames.txt

Yep, that's four backslashes in a row. This creates the required changes: a filename like bla$1$2.class would then change to bla\$1\$2.class. This I can then insert into the full pipeline:

for i in $(diff -r old new 2>/dev/null | grep "Only in old" | cut -d "/" -f 3- | sed "s/: /\//g" | sed "s/\\$/\\\\$/g"; do echo "rm -f $i" >> REMOVEOLDFILES.sh; done

Alternative to solve the background problem

chepner posted an alternative to solve the backround problem by simply adding single-quotes around the filenames for the output. This way, the $-signs are not read as variables by bash when executing the script and the files are also properly removed:

for i in $(diff -r old new 2>/dev/null | grep "Only in old" | cut -d "/" -f 3- | sed "s/: /\//g"); do echo "rm -f '$i'" >> REMOVEOLDFILES.sh; done

(note the changed echo "rm -f '$i'" in that line)

like image 166
Captain Ahab Avatar answered Sep 17 '22 17:09

Captain Ahab