Handling CR line endings in Lua

Question

I'm trying to read a file with CR line endings using the file:read method which seems to be acting up for some reason. The file contents look like this:

ABCDEFGH
12345
##
6789

I want it to behave consistently with all types of line endings. Every time I try to read the file, it returns the last line in the file concatenated with the any trailing characters from the previous lines that have a greater position than the position of the last character in the last line. Here's what I mean:

> file=io.open("test.lua", "rb")
> function re_read(openFile)
     openFile:seek("set");
     return openFile:read("*a");
  end
> =re_read(file) -- With CR
67895FGH

> =re_read(file) -- With CRLF
ABCDEFGH
12345
##
6789

> =re_read(file) -- with LF
ABCDEFGH
12345
##
6789

>

As you can see, the string being returned is the last string plus 5 in the previous line and plus FGH from the first line. Any lines shorter than the last line are skipped.

My goal is to use the file:line() method to read the file line by line. My hope is that if a 'fix' for file:read is found then it can be applied to file:lines().

Answer 1

In the case with CR only, re_read actually works as expected: it returns the lines separated by CR. But when the interpreter displays it, it interprets the CR characters as "go back to the beginning of the line". So here is how the result changes line by line:

ABCDEFGH
12345FGH
##345FGH
67895FGH

EDIT: here it is character by character, with a "virtual cursor" (|).

|
A|
AB|
ABC|
ABCD|
ABCDEF|
ABCDEFGH|
|ABCDEFGH
1|BCDEFGH
12|CDEFGH
123|DEFGH
1234|EFGH
12345|FGH
|12345FGH
#|2345FGH
##|345FGH
|##345FGH
6|#345FGH
67|345FGH
678|45FGH
6789|5FGH

Proof:

> s = "ABCDEFGH\r12345\r##\r6789"
> =s
67895FGH

Answer 2

You could normalize your line endings with gsub then iterate over the product with gmatch.

local function cr_lines(s)
    return s:gsub('\r\n?', '\n'):gmatch('(.-)\n')
end

local function cr_file_lines(filename)
    local f = io.open(filename, 'rb')
    local s = f:read('*a')
    f:close()
    return cr_lines(s)
end

for ln in cr_file_lines('test.txt') do
    print(ln)
end