Ruby open-uri redirect forbidden

Question

I have this simple html parser(for learning purposes) that I have been working on.:

require 'open-uri' puts "Enter URL to parse HTML: " url = gets.chomp puts "Enter tag to parse from: " tag = gets.chomp response = open(url).read title1 = response.index(tag) title2 = response.index(tag.insert(1,'/')) -1 result = response[(title1 + tag.length - 1)..title2] print result  

and when I input http://twitter.com, I get this error message:

ERROR: `open_loop': redirection forbidden: http://twitter.com -> https://twitter.com/ (RuntimeError) from /usr/local/rvm/rubies/ruby-2.1.4/lib/ruby/2.1.0/open-uri.rb:149:in `open_uri' from /usr/local/rvm/rubies/ruby-2.1.4/lib/ruby/2.1.0/open-uri.rb:704:in `open' from /usr/local/rvm/rubies/ruby-2.1.4/lib/ruby/2.1.0/open-uri.rb:34:in `open' from /home/ubuntu/workspace/htmlparse.rb:6:in `<main>'  

Any advise or help? Im new to Ruby and I am aware of other html parsing modules, but Im doing this to learn Ruby basics. Thanks.

Answer 1

Have a look at the open_uri_redirections gem.

It patches Ruby's OpenURI to allow redirections from HTTP to HTTPS or the other way around.

Answer 2

You can also catch the exception and then try it again with 'https' url.

url = "http://classic.ona.io/api/v1/files/3538545?filename=gringgo/attachments/1485229166168.jpg"  uri = URI.parse(url) tries = 3  begin   uri.open(redirect: false) rescue OpenURI::HTTPRedirect => redirect   uri = redirect.uri # assigned from the "Location" response header   retry if (tries -= 1) > 0   raise end 

Source: https://twin.github.io/improving-open-uri/