How to make an HTTP GET with modified headers?

Question

What is the best way to make an HTTP GET request in Ruby with modified headers?

I want to get a range of bytes from the end of a log file and have been toying with the following code, but the server is throwing back a response saying that "it is a request that the server could not understand" (the server is Apache).

require 'net/http'
require 'uri'

#with @address, @port, @path all defined elsewhere

httpcall = Net::HTTP.new(@address, @port)

headers = {
  'Range' => 'bytes=1000-'
}

resp, data = httpcall.get2(@path, headers)

1. Is there a better way to define headers in Ruby?
2. Does anyone know why this would be failing against Apache? If I do a get in a browser to http://[address]:[port]/[path] I get the data I am seeking without issue.

Answer 1

Created a solution that worked for me (worked very well) - this example getting a range offset:

require 'uri'
require 'net/http'

size = 1000 #the last offset (for the range header)
uri = URI("http://localhost:80/index.html")
http = Net::HTTP.new(uri.host, uri.port)
headers = {
    'Range' => "bytes=#{size}-"
}
path = uri.path.empty? ? "/" : uri.path

#test to ensure that the request will be valid - first get the head
code = http.head(path, headers).code.to_i
if (code >= 200 && code < 300) then

    #the data is available...
    http.get(uri.path, headers) do |chunk|
        #provided the data is good, print it...
        print chunk unless chunk =~ />416.+Range/
    end
end

Answer 2

If you have access to the server logs, try comparing the request from the browser with the one from Ruby and see if that tells you anything. If this isn't practical, fire up Webrick as a mock of the file server. Don't worry about the results, just compare the requests to see what they are doing differently.

As for Ruby style, you could move the headers inline, like so:

httpcall = Net::HTTP.new(@address, @port)

resp, data = httpcall.get2(@path, 'Range' => 'bytes=1000-')

Also, note that in Ruby 1.8+, what you are almost certainly running, Net::HTTP#get2 returns a single HTTPResponse object, not a resp, data pair.