Lua : Fetch a webpage

Question

I want to fetch a webpage and get the result in a string, but I don't know how to do it. I search online and didn't find how to do it.

Answer 1

I'd simply use Lua Socket which comes with an http submodule. You can simply use http.request to get a webpage into whatever container you'd want (default is string, but you can use a table, a file, stdio, ... using ltn12 filters and sinks).

As an example:

local http=require'socket.http'
local body, statusCode, headers, statusText = http.request('http://w3.impa.br/~diego/software/luasocket/http.html')
print('statusCode ', statusCode)
print('statusText ', statusText)
print('headers ')
for index,value in pairs(headers) do
    print("\t",index, value)
end
print('body',body)

Answer 2

If you can't find an exact http client library, you could implement on yourself, or build on someone else's work.

In that link, although it is called libhttpd, but the author clearly states that it can be used for anything. Looks like a more usable wrapper around lua sockets.

Answer 3

if you don't have socket (like me), but you have the http library/module, then you could try this:

http.get("https://nodemcu.readthedocs.io/en/master/en/modules/http/", nil, function(code, data)
    if (code ~= 200) then
        print("HTTP request failed")
    else
        print(code, data)
    end
end)

it works for me

you can find more info in the docs https://nodemcu.readthedocs.io/en/master/en/modules/http/#httpget