Can we use selenium-webdriver and nokogiri together?

Question

I used Nokogiri as below:

require 'nokogiri'
require 'open-uri'

# Get a Nokogiri::HTML::Document for the page we’re interested in...

doc = Nokogiri::HTML(open('http://www.google.com/search?q=sparklemotion'))

But my bad,due to company firewall may be, I am getting error:

C:/Ruby193/lib/ruby/1.9.1/net/http.rb:762:in `initialize': getaddrinfo: No such
host is known.  (SocketError)

Thus I thought that I would use selenium-webdriver to navigate and nokogiri to work on webpage source html.

require "rubygems"
require "selenium-webdriver"

driver = Selenium::WebDriver.for :firefox
driver.get "http://www.google.com/search?q=sparklemotion"

So here how would I give webpage content(html) to nokogiri?

Please suggest me here.

Answer 1

You can get the source of the page from selenium-webdriver by using the page_source method:

driver.page_source

So your script could be:

require 'selenium-webdriver'
require 'nokogiri'

driver = Selenium::WebDriver.for :firefox
driver.get "http://www.google.com/"

doc = Nokogiri::HTML(driver.page_source)
# Do whatever with nokogiri

That said, I do not know why you would want to use nokogiri instead of just selenium-webdriver.