Reading file line by line in Lua

Question

As per Lua documentation, file:read("*l") reads next line skipping end of line.

Note:- "*l": reads the next line skipping the end of line, returning nil on end of file. This is the default format

Is this documentation right? Because file:read("*l") reads the current line,instead of next line or my understanding is wrong? Pretty confusing...

Answer 1

Lua manages files using the same model of the underlying C implementation (this model is used also by other programming languages and it is fairly common). If you are not familiar with this way of looking at files, the terminology could be unclear, indeed.

In this model a file is represented as a stream of bytes having a so called current position. The current position is a sort of conceptual pointer to the first byte in the file that will be read or written by the next I/O operation. When you open a file for reading, a new stream is set-up so that its current position is the beginning of the file, i.e. the current position "points" to the first byte in the file.

In Lua you manage streams through so-called file handles, which are a sort of intermediaries for the underlying streams. Any operation you perform using the handle is carried over to the corresponding stream.

Lua io.open opens a file, associates a C stream with it and returns a file handle that represents that stream:

local file_handle = io.open( "myfile.txt" ) -- file opened for reading

Therefore, if you perform any operation that reads some bytes (usually interpreted as characters, if you work with text files) those are read from the stream and for each byte read the current position of the stream advances by one, pointing each time to the next byte to be read.

Lua documentation implies this model. Thus when it says next line, it means that the input operation will read all characters in the stream starting from the current position until an end-of-line character is found.

Note that if you look at text files as a sequence of lines you could be misled, since you could think of a "current line" and a "next line". That would be an higher level model compared to the C model. There is no "current line" in C. In C text files are nothing more than a sequence of bytes where some special characters (end-of-line characters) undergo some special treatment (which is mostly implementation-dependent) and are used by some C standard functions as line terminators, i.e. as marks to detect when stop reading characters.

Another source of confusion for newbies or people coming from higher level languages is that in C, for an historical accident, bytes are handled as characters (the basic data type to handle single bytes is char, which is the smallest numeric type in C!). Therefore for people with a C background it is natural to think of bytes as characters and vice versa.

Although Lua is a much higher level language than C, its close relationship with C (it was designed to be easily interfaced with C code) makes it inherit part of this C "bytes-as-characters" approach. In fact, for example, Lua strings can hold arbitrary bytes and can be used to process raw binary data.

Answer 2

Like Lorenso said above, read starts at the current file position and reads from that position some portion of the file. How much of the file it reads depends on read instruction. For reference, in Lua 5.3:

• "*all" : reads to the end of the file
• "*line" : reads from the current position to the end of the line. The end of the line is marked by a special character usually denoted LfCr (Line feed, carriage return )
• "*number" : reads a number, that is, it will read up to the end of what it recognizes in the text as a number, stopping at, for example, a comma ",".
• num : reads a string with up to num characters

Here's an example that reads a file with a list of numbers into an array (a table), then returns the array. (Just change the "*number" to "*line" and it would read a file line by line):

function read_array(file)
  local arr = {}
  local handle  = assert( io.open(file,"r") )
  local value = handle:read("*number")
  while value do
    table.insert( arr, value )
    value = handle:read("*number")
  end
  handle:close()
  return arr
end