Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Using sed to replace a string with the contents of a variable, even if it's an escape character

Tags:

string

bash

sed

I'm using sed -e "s/\*DIVIDER\*/$DIVIDER/g" to replace *DIVIDER* with a user-specified string, which is stored in $DIVIDER. The problem is that I want them to be able to specify escape characters as their divider, like \n or \t. When I try this, I just end up with the letter n or t, or so on.

Does anyone have any ideas on how to do this? It will be greatly appreciated!

EDIT: Here's the meat of the script, I must be missing something.

curl --silent "$URL" > tweets.txt

if [[ `cat tweets.txt` == *\<error\>* ]]; then
    grep -E '(error>)' tweets.txt | \
    sed -e 's/<error>//' -e 's/<\/error>//' |
    sed -e 's/<[^>]*>//g' |

head $headarg | sed G | fmt

else
    echo $REPLACE | awk '{gsub(".", "\\\\&");print}'
    grep -E '(description>)' tweets.txt | \
    sed -n '2,$p' | \
    sed -e 's/<description>//' -e 's/<\/description>//' |
    sed -e 's/<[^>]*>//g' |
    sed -e 's/\&amp\;/\&/g' |
    sed -e 's/\&lt\;/\</g' |
    sed -e 's/\&gt\;/\>/g' |
    sed -e 's/\&quot\;/\"/g' |
    sed -e 's/\&....\;/\?/g' |
    sed -e 's/\&.....\;/\?/g' |
    sed -e 's/^  *//g' |
    sed -e :a -e '$!N;s/\n/\*DIVIDER\*/;ta' |   # Replace newlines with *divider*.
    sed -e "s/\*DIVIDER\*/${DIVIDER//\\/\\\\}/g" |          # Replace *DIVIDER* with the actual divider.

    head $headarg | sed G
fi

The long list of sed lines are replacing characters from an XML source, and the last two are the ones that are supposed to replace the newlines with the specified character. I know it seems redundant to replace a newline with another newline, but it was the easiest way I could come up with to let them pick their own divider. The divider replacement works great with normal characters.

like image 868
vilhalmer Avatar asked Feb 13 '10 15:02

vilhalmer


People also ask

How do you escape special characters in sed?

Put a backslash before $. */[\]^ and only those characters (but not inside bracket expressions).

How do you replace a variable in a file using sed?

For replacing a variable value using sed, we first need to understand how sed works and how we can replace a simple string in any file using sed. In this syntax, you just need to provide the string you want to replace at the old string and then the new string in the inverted commas.

How can I replace text after a specific word using sed?

Find and replace text within a file using sed command Use Stream EDitor (sed) as follows: sed -i 's/old-text/new-text/g' input.txt. The s is the substitute command of sed for find and replace. It tells sed to find all occurrences of 'old-text' and replace with 'new-text' in a file named input.txt.


2 Answers

You can use bash to escape the backslash like this:

sed -e "s/\*DIVIDER\*/${DIVIDER//\\/\\\\}/g"

The syntax is ${name/pattern/string}. If pattern begins with /, every occurence of pattern in name is replaced by string. Otherwise only the first occurence is replaced.

like image 189
tangens Avatar answered Oct 02 '22 09:10

tangens


Maybe:

case "$DIVIDER" in
(*\\*) DIVIDER=$(echo "$DIVIDER" | sed 's/\\/\\\\/g');;
esac

I played with this script:

for DIVIDER in 'xx\n' 'xxx\\ddd' "xxx"
do
    echo "In:  <<$DIVIDER>>"
    case "$DIVIDER" in     (*\\*) DIVIDER=$(echo "$DIVIDER" | sed 's/\\/\\\\/g');;
    esac
    echo "Out: <<$DIVIDER>>"
done

Run with 'ksh' or 'bash' (but not 'sh') on MacOS X:

In:  <<xx\n>>
Out: <<xx\\n>>
In:  <<xxx\\ddd>>
Out: <<xxx\\\\ddd>>
In:  <<xxx>>
Out: <<xxx>>
like image 28
Jonathan Leffler Avatar answered Oct 02 '22 11:10

Jonathan Leffler