Does io.lines() stream or slurp the file?

Question

For algorithms that support line by line processing, lua documentation suggests that using io.lines() is more efficient than io:read("*line") in a while loop.

The call io.read("*line") returns the next line from the current input file, without the newline character. (...) However, to iterate on a whole file line by line, we do better to use the io.lines iterator. (21.1 – The Simple I/O Model)

I can imagine three possible reasons that the io.lines() call is preferred.

1. The iterator is more efficient than the while loop
2. The file reading is handled more efficiently
3. It's easier to read/maintain the code

The lua documentation also promotes slurping files

(Y)ou should always consider the alternative of reading the whole file with option "*all" from io.read and then using gfind to break it up (21.1 – The Simple I/O Model)

Hypothesis: io:read("*line") streams the file. If slurping is more efficient in lua, and io.lines() slurps the file, then io.lines() might be more efficient for that reason.

However, the unofficial Lua FAQ has the following to say about io.lines()

Note that it is an iterator, this does not bring the whole file into memory initially.

This suggests streaming instead of slurping.

TLDR Does io.lines() ever hold the whole file in memory or does it only hold one line in memory at a time? Is its memory usage different than io:read("*line") in a while loop?

Answer 1

io.lines() does not hold the whole file in memory: it reads the file one line at a time, not the whole file at once. For that, use io.read("*all").