Retrieve contents of URL as string

Question

For tedious reasons to do with Hpricot, I need to write a function that is passed a URL, and returns the whole contents of the page as a single string.

I'm close. I know I need to use OpenURI, and it should look something like this:

require 'open-uri'
open(url) {
  # do something mysterious here to get page_string
}
puts page_string

Can anyone suggest what I need to add?

Answer 1

You can do the same without OpenURI:

require 'net/http'
require 'uri'

def open(url)
  Net::HTTP.get(URI.parse(url))
end

page_content = open('http://www.google.com')
puts page_content

Or, more succinctly:

Net::HTTP.get(URI.parse('http://www.google.com'))

Answer 2

The open method passes an IO representation of the resource to your block when it yields. You can read from it using the IO#read method

open([mode [, perm]] [, options]) [{|io| ... }] 
open(path) { |io| data = io.read }

Answer 3

require 'open-uri'
open(url) do |f|
  page_string = f.read
end

See also the documentation of IO class