Ruby - net/http - following redirects

Question

I've got a URL and I'm using HTTP GET to pass a query along to a page. What happens with the most recent flavor (in net/http) is that the script doesn't go beyond the 302 response. I've tried several different solutions; HTTPClient, net/http, Rest-Client, Patron...

I need a way to continue to the final page in order to validate an attribute tag on that pages html. The redirection is due to a mobile user agent hitting a page that redirects to a mobile view, hence the mobile user agent in the header. Here is my code as it is today:

require 'uri' require 'net/http'  class Check_Get_Page      def more_http         url = URI.parse('my_url')         req, data = Net::HTTP::Get.new(url.path, {         'User-Agent' => 'Mozilla/5.0 (iPhone; U; CPU iPhone OS 4_3_2 like Mac OS X; en-us) AppleWebKit/533.17.9 (KHTML, like Gecko) Version/5.0.2 Mobile/8H7 Safari/6533.18.5'         })         res = Net::HTTP.start(url.host, url.port) {|http|         http.request(req)             }         cookie = res.response['set-cookie']         puts 'Body = ' + res.body         puts 'Message = ' + res.message         puts 'Code = ' + res.code         puts "Cookie \n" + cookie     end  end  m = Check_Get_Page.new m.more_http 

Any suggestions would be greatly appreciated!

Answer 1

To follow redirects, you can do something like this (taken from ruby-doc)

Following Redirection

require 'net/http' require 'uri'  def fetch(uri_str, limit = 10)   # You should choose better exception.   raise ArgumentError, 'HTTP redirect too deep' if limit == 0    url = URI.parse(uri_str)   req = Net::HTTP::Get.new(url.path, { 'User-Agent' => 'Mozilla/5.0 (etc...)' })   response = Net::HTTP.start(url.host, url.port, use_ssl: true) { |http| http.request(req) }   case response   when Net::HTTPSuccess     then response   when Net::HTTPRedirection then fetch(response['location'], limit - 1)   else     response.error!   end end  print fetch('http://www.ruby-lang.org/') 

Answer 2

Given a URL that redirects

url = 'http://httpbin.org/redirect-to?url=http%3A%2F%2Fhttpbin.org%2Fredirect-to%3Furl%3Dhttp%3A%2F%2Fexample.org' 

A. Net::HTTP

begin   response = Net::HTTP.get_response(URI.parse(url))   url = response['location'] end while response.is_a?(Net::HTTPRedirection) 

Make sure that you handle the case when there are too many redirects.

B. OpenURI

open(url).read 

OpenURI::OpenRead#open follows redirects by default, but it doesn't limit the number of redirects.