Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Render HTTP Response(HTML content) in selenium webdriver(browser)

I am using Requests module to send GET and POST requests to websites and then processing their responses. If the Response.text meets a certain criteria, I want it to be opened up in a browser. To do so currently I am using selenium package and resending the request to the webpage via the selenium webdriver. However, I feel it's inefficient as I have already obtained the response once, so is there a way to render this obtained Response object directly into the browser opened via selenium ?

EDIT A hacky way that I could think of is to write the response.text to a temporary file and open that in the browser. Please let me know if there is a better way to do it than this ?

like image 262
bawejakunal Avatar asked Apr 22 '16 05:04

bawejakunal


People also ask

How do I get HTML response in Selenium?

You can retrieve the HTML source of an URL with the code shown below. It first starts the web browser (Firefox), loads the page and then outputs the HTML code. The code below starts the Firefox web rbowser, opens a webpage with the get() method and finally stores the webpage html with browser.

How do I get HTML text in Selenium?

To get the HTML source of a WebElement in Selenium WebDriver, we can use the get_attribute method of the Selenium Python WebDriver. First, we grab the HTML WebElement using driver element locator methods like (find_element_by_xpath or find_element_by_css_selector).

Can we get the HTTP response code in Selenium with Java?

We can get the HTTP response code in Selenium webdriver with Java. Some of the response codes are – 2xx, 3xx, 4xx and 5xx. The 2xx response code signifies the proper condition, 3xx represents redirection, 4xx shows resources cannot be identified and 5xx signifies server problems.

Can Selenium parse HTML?

Selenium is a library which will interface with the browser, allow for the site to render, and then allow you to retrieve the data from the browser's DOM. If you need to, you can script the browser to click on various links to load HTML partials that can also be parsed to get additional detail.


1 Answers

To directly render some HTML with Selenium, you can use the data scheme with the get method:

from selenium import webdriver
import requests

content = requests.get("http://stackoverflow.com/").content

driver = webdriver.Chrome()
driver.get("data:text/html;charset=utf-8," + content)

Or you could write the page with a piece of script:

from selenium import webdriver
import requests

content = requests.get("http://stackoverflow.com/").content

driver = webdriver.Chrome()
driver.execute_script("""
  document.location = 'about:blank';
  document.open();
  document.write(arguments[0]);
  document.close();
  """, content)
like image 198
Florent B. Avatar answered Oct 19 '22 22:10

Florent B.