I'm using this code to scraping external html files
link = URI.parse(url) request = Net::HTTP::Get.new(link.path) response = Net::HTTP.start(link.host, link.port) {|http| http.request(request) }
Works great but with slowed web pages sometimes responds timeout, so I need set a timeout limit per connection. Any idea?
You need to set the read_timeout attribute.
link = URI.parse(url)
request = Net::HTTP::Get.new(link.path)
begin
response = Net::HTTP.start(link.host, link.port) {|http|
http.read_timeout = 100 #Default is 60 seconds
http.request(request)
}
rescue Net::ReadTimeout => e
puts e.message
end
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With