Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Optimize shell script for multiple sed replacements

Tags:

bash

shell

sed

I have a file containing a list of replacement pairs (about 100 of them) which are used by sed to replace strings in files.

The pairs go like:

old|new
tobereplaced|replacement
(stuffiwant).*(too)|\1\2

and my current code is:

cat replacement_list | while read i
do
    old=$(echo "$i" | awk -F'|' '{print $1}')    #due to the need for extended regex
    new=$(echo "$i" | awk -F'|' '{print $2}')
    sed -r "s/`echo "$old"`/`echo "$new"`/g" -i file
done

I cannot help but think that there is a more optimal way of performing the replacements. I tried turning the loop around to run through lines of the file first but that turned out to be much more expensive.

Are there any other ways of speeding up this script?

EDIT

Thanks for all the quick responses. Let me try out the various suggestions before choosing an answer.

One thing to clear up: I also need subexpressions/groups functionality. For example, one replacement I might need is:

([0-9])U|\10  #the extra brackets and escapes were required for my original code

Some details on the improvements (to be updated):

  • Method: processing time
  • Original script: 0.85s
  • cut instead of awk: 0.71s
  • anubhava's method: 0.18s
  • chthonicdaemon's method: 0.01s
like image 900
Reuben L. Avatar asked Aug 29 '14 06:08

Reuben L.


1 Answers

You can use sed to produce correctly -formatted sed input:

sed -e 's/^/s|/; s/$/|g/' replacement_list | sed -r -f - file
like image 140
chthonicdaemon Avatar answered Sep 19 '22 20:09

chthonicdaemon