Lua - io.open() only up to 2 GB?

Question

I am using a Lua script to determine the file size:

local filesize=0
local filePath = "somepath.bin"
local file,msg = io.open(filePath, "r")
if file then
    filesize=file:seek("end")
    file:close()
    filePresent = true
end

However, this only seem to work for files up to 2GB. For larger files filesize is always nil. Is there any limitation on io.open? And if so, how could I work around this?

Running Lua 5.1.4 on Windows Server 2008 R2 64bit

Answer 1

The problem is not in io.open, but file:seek. You can check the error like this:

filesize, err = file:seek("end")
if not filesize then
    print(err)
end

The error message is probably Invalid argument. That's because for files larger than 2GB, its size is over what 32-bit long can hold, which causes the C function fseek fail to work.

In POSIX systems, Lua uses fseeko which takes the size of off_t instead of long in fseek. In Windows, there's a _fseeki64 which I guess does similar job. If these are not available, fseek is used, and it would cause the problem.

---

The relevant source is liolib.c(Lua 5.2). As @lhf points out, in Lua 5.1, fseek is always used (source). Upgrading to Lua 5.2 could possibly solve the problem.

Answer 2

Internally, Lua uses the ISO C function long int ftell(FILE *stream); to determine the return value for file:seek(). A long int is always 32 bits on Windows, so you are out of luck here. If you can, you should use some external library to detect the file size -- I recommend luafilesystem.