sed behavior with & (ampersand)

Question

Trying to clean up some xml text that looks like

Forest & Paper Products Manufacturing

with a sed command like

 sed "s/ \& / & /"

but once sed gets done with the file, my output looks like

Forest & amp; Paper Products Manufacturing

Can't figure out why sed is putting a space after the &

I can solve this with:

sed "s/ \& / \& /"

But why do I need to quote the & with a \ prefix

Answer 1

in the replacement part, & indicates the matched string \0. So if you want to have literal & in replacement part, you have to escape it.

E.g

kent$ (master|✔) echo "foo"|sed 's/.*/&_bar/'   
foo_bar

kent$ (master|✔) echo "foo"|sed 's/.*/\&_bar/' 
&_bar

related info taken from sed's INFO page:

   The REPLACEMENT can contain `\N' (N being a number from 1 to 9,
 inclusive) references, which refer to the portion of the match which
 is contained between the Nth `\(' and its matching `\)'.  Also, the
 REPLACEMENT can contain unescaped `&' characters which reference the
 whole matched portion of the pattern space

and

To include a literal `\', `&', or newline in the final replacement, be   
 sure to precede the desired `\', `&', or newline in the REPLACEMENT with 
 a `\'.