sed replace multiple blank lines with single blank line

Question

I have written a sed script, which replaces multiple blank lines with a single one, but it is not working as supposed to. I will be thankful to everybody, who can explain me why. Please do not refer me to working examples, I am familiar with Google. I just want to understand how sed works.

The code is

sed ':a;/^\n*$/{N;ba};s/^\n\n*/\n/' input_file

so the logic is simple: when sed reads the line and it is either blank or has several newline symbols (this is /^*\n$ condition), I tell sed to append the next line to pattern space. as soon as a non-blank line is found, the substitution s/^\n\n*/\n/ is done.

Everything works fine except the cases when I have blank lines in the end of the file. These blanks are not replaced with a single blank and I do not understand why.

Any ideas?

Answer 1

The problem is sed getting EOF while reading next line.

Your command is getting completed at the time of reading next line using N. Because, While reading the next line, sed getting EOF . So, It won't process s/^\n\n*/\n/ substitution. That is why, You are not able to remove sequence of empty lines which has appeared in end of file.

My solution is:

sed ':a; /^\n*$/{ s/\n//; N;  ba};' yourfile

Answer 2

Instead of \n\n*, you could use \n+

By the way, this is what it would look like as a Perl one-liner:

perl -0777 -pe 's/\n+/\n/g' yourfile

Or, for the same result but fewer substitutions:

perl -0777 -pe 's/\n\n+/\n/g' yourfile

Or, even more magical, try this compact solution suggested by @hwnd:

perl -00 -pe ''