Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Replace multiple lines using sed

Tags:

sed

I thought I understood sed but I guess not. I have the following two files, in which I want to replace the "why" and "huh" lines with one different line. No whitespace at all.

test.txt:

hi
why
huh
hi
why
huh

test2.txt:

1
hi
why
huh
hi
why
huh

The following two commands give the following results:

sed "N; s/<why\/>\n<huh\/>/yo/g" test.txt > out.txt

out.txt:
hi
why
huh
hi
yo

sed "N; s/<why\/>\n<huh\/>/yo/g" test2.txt > out2.txt

out2.txt:
1
hi
yo
hi
why
huh

What am I not understanding about sed? Why don't both output files contain the following:

hi
yo
hi
yo
like image 500
user1145925 Avatar asked Jun 27 '12 19:06

user1145925


People also ask

Can sed replace multiple lines?

Different commands exist in Linux to replace multiple lines of a file. `sed` command is one of them to do this type of task.

How do you match multiple lines in sed?

By using N and D commands, sed can apply regular expressions on multiple lines (that is, multiple lines are stored in the pattern space, and the regular expression works on it): $ cat two-cities-dup2.

How do you replace a line in a file using sed?

Find and replace text within a file using sed command Use Stream EDitor (sed) as follows: sed -i 's/old-text/new-text/g' input.txt. The s is the substitute command of sed for find and replace. It tells sed to find all occurrences of 'old-text' and replace with 'new-text' in a file named input.txt.

What does sed n do?

N reads the next line into pattern space. [Now there are 2 lines in pattern space.] If there is no next line to read then sed exits here. [ie: In case of odd lines, sed exits here - and hence the last line is swallowed without printing.]


3 Answers

Your expression is almost correct, but it has two problems:

  • If you want to match why as a word, you should put \< and \> around it. You did put just < and \/> around it. So, the first correction is:

    $ sed 'N; s/\<why\>\n\<huh\>/yo/g' test.txt
    
  • But it will not work, either:

    $ sed 'N; s/\<why\>\n\<huh\>/yo/g' test.txt
    hi
    why
    huh
    hi
    yo
    

    Why does it replace only the second pair of lines? Well, in the first line, the N command will concatenate why to hi, leaving in the pattern space the string hi\nwhy. This string is not matched by the s/// command, so the line is just printed. Next time, you have the string huh in the pattern space and concatenate hi to it. Just in the next line you will have why\nhuh in the pattern space to be replaced.

    The solution is to concatenate the next line only when your current line is why, using the address /^why$/:

    $ sed '/^why$/ {N; s/\<why\>\n\<huh\>/yo/g}' test.txt
    hi
    yo
    hi
    yo
    
like image 110
brandizzi Avatar answered Oct 03 '22 17:10

brandizzi


The reason why it didn't replace both pairs of lines is explained beautifully in brandizzi's answer.

However, if we take one step further. Say we have the following file and we want to replace "apple\njuice" with "pure\nmilk".

test.txt

water water water water 
water water water apple
juice water water apple
juice water water water

The pattern filter way would not work.

sed '/apple/ {N; s/apple\njuice/pure\nmilk/g}' test.txt 

water water water water
water water water pure
milk water water apple
juice water water water

Because the 2nd apple from test.txt, which has been concatenated to the previous line by N, didn't get caught by pattern filter.

One solution I can think of is to use branch to concatenate all lines and then do the replacement.

sed ':a;N;$!ba;s/apple\njuice/pure\nmilk/g' test.txt 

water water water water
water water water pure
milk water water pure
milk water water water

It looks dumb, but I haven't think of a better way yet.

like image 24
Cz Zhang Avatar answered Oct 03 '22 18:10

Cz Zhang


This should work for test.txt file:

sed '/hi/! { N ; s/why\nhuh/yo/ }' test.txt

It means:

When not found hi in a line (it will be why), read next one and substitute all it with yo. Otherwise print directly (when hi).

Output:

hi
yo
hi
yo
like image 41
Birei Avatar answered Oct 03 '22 16:10

Birei