What does the 'N' command do in sed?

Question

It looks like the 'N' command works on every other line:

$ cat in.txt 
 a
 b
 c
 d
$ sed '=;N' in.txt 
1
 a
 b
3
 c
 d

Maybe that would be natural because command 'N' joins the next line and changes the current line number. But (I saw this here):

$ sed 'N;$!P;$!D;$d' thegeekstuff.txt

The above example deletes the last two lines of a file. This works not only for even-line-numbered files but also for odd-line-numbered files. In this example 'N' command runs on every line. What's the difference?

And could you tell me why I cannot see the last line when I run sed like this:

# sed N odd-lined-file.txt

Answer 1

Excerpt from info sed:

`sed' operates by performing the following cycle on each lines of
input: first, `sed' reads one line from the input stream, removes any
trailing newline, and places it in the pattern space.  Then commands
are executed; each command can have an address associated to it:
addresses are a kind of condition code, and a command is only executed
if the condition is verified before the command is to be executed.
...
When the end of the script is reached, unless the `-n' option is in
use, the contents of pattern space are printed out to the output
stream,
...
Unless special commands (like 'D') are used, the pattern space is
deleted between two cycles

...

`N'
     Add a newline to the pattern space, then append the next line of
     input to the pattern space.  If there is no more input then `sed'
     exits without processing any more commands.

...

`D'
     Delete text in the pattern space up to the first newline.  If any
     text is left, restart cycle with the resultant pattern space
     (without reading a new line of input), otherwise start a normal
     new cycle.

This should pretty much resolve your query. But still I will try to explain your three different cases:

CASE 1:

1. sed reads a line from input. [Now there is 1 line in pattern space.]
2. = Prints the current line no.
3. N reads the next line into pattern space.[Now there are 2 lines in pattern space.]
   • If there is no next line to read then sed exits here. [ie: In case of odd lines, sed exits here - and hence the last line is swallowed without printing.]
4. sed prints the pattern space and cleans it. [Pattern space is empty.]
5. If EOF reached sed exits here. Else Restart the complete cycle from step 1. [ie: In case of even lines, sed exits here.]

Summary: In this case sed reads 2 lines and prints 2 lines at a time. Last line is swallowed it there are odd lines (see step 3).

CASE 2:

1. sed reads a line from input. [Now there is 1 line in pattern space.]
2. N reads the next line into pattern space. [Now there are 2 lines in pattern space.]
   • If it fails exit here. This occurs only if there is 1 line.
3. If its not last line($!) print the first line(P) from pattern space. [The first line from pattern space is printed. But still there are 2 lines in pattern space.]
4. If its not last line($!) delete the first line(D) from pattern space [Now there is only 1 line (the second one) in the pattern space.] and restart the command cycle from step 2. And its because of the command D (see the excerpt above).
5. If its last line($) then delete(d) the complete pattern space. [ie. reached EOF ] [Before beginning this step there were 2 lines in the pattern space which are now cleaned up by d - at the end of this step, the pattern space is empty.]
6. sed automatically stops at EOF.

Summary: In this case :

• sed reads 2 lines first.
• if there is next line available to read, print the first line and read the next line.
• else delete both lines from cache. This way it always deletes the last 2 line.

CASE 3:
Its the same case as CASE:1, just remove the Step 2 from it.