Why do gedit and vim hide the final newline from the user?

Question

Let's assume we have two text documents:

1. Our first file contains "hi" as text.
2. Our second file contains "hi" as text.

When we open these two files in gedit, vi, or vim, the two files are visually identical in every way.

However, when we run xxd on the files, we get the following:

1. Hex content of our first file reads: 6869
2. Hex content of our second file reads: 6869 0a

Aha! There's an invisible newline. In vim, if we were paying close enough attention to the status bar and happen to understand what [noeol] means, then we might pick up on this, but in gedit, the two files open exactly the same!

In a small survey, when I asked people to distinguish between the two files using only gedit or vim, they failed 100% of the time. When I asked them to accomplish the same task using leafpad or emacs, they succeeded 100% of the time.

I understand that vi and gedit want to add a newline to every file they create (and I acknowledge that there are probably advantages to this). What I don't understand is why gedit and vim think visually hiding this newline from their users is beneficial? Especially when this kind of behavior is potentially extremely destructive...

(Take for instance the two C-programmers who see the contents of these two files the same in their vi/gedit text editor, and then assuming what they see is what they get, go on to write the contents into the array char greeting[2]. The first programmer writing the first file - although a bit sloppy with his code - goes on to fame and fortune, but the second programmer writing the second file, dies in miserable poverty, confused and bewildered by this invisible (and preventable) stack overflow.)

So pray tell, what are the advantages of having text editors like vim and gedit add invisible newlines at the end of each document they create, and then proceed to hide those newlines from the user, so that the true contents of these files are only visibly detectable using other text editors?

Answer 1

That you perceive the last line as "visually hidden" stems from the opposite viewpoints.

Many people (especially with a Unix background) argue that text files should always end with a newline (e.g. see Why should text files end with a newline?), and that any text editor that allows to create files without it is fundamentally broken. From that perspective, that there is no additional empty line in Vim is only consistent with this world view. The file appears in the editor as it would when it is cated into the terminal.

From Vim's point of view, a file with a missing newline is heretical and defective; that's why it is indicated with the [noeol] message on load (and it is quite hard to edit the file while keeping the missing newline; though I've written the PreserveNoEOL plugin to aid with that).

Answer 2

I believe this is a convention, that is carried over from vi. One advantage of doing so, is that when concatenating files, you can still distinguish where each file ended.

Some more context is provided in this thread on the vim_use mailinglist