Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Getting meta tags from a page source using Selenium Python

I'm trying to fetch the data from the URL https://play.google.com/store/apps/details?id=com.teslacoilsw.launcher&hl=en and fetch the below data

   <meta content="3.99" itemprop="price"> 

I used the following code implemented in Python to fetch but it failed.

    browser = webdriver.Firefox() # Get local session of firefox
    browser.get(sampleURL) # Load page
    assert "Google Play" in browser.title
    priceValue = browser.find_element_by_xpath("//div[@itemprop='price']")#
    print priceValue.text

But it says it can't find the xpath of value price. Any idea why?

EDIT

priceValue = browser.find_element_by_xpath("//meta[@itemprop='price']")
print priceValue.text

I get empty string

like image 839
Siddharthan Asokan Avatar asked Sep 27 '13 18:09

Siddharthan Asokan


People also ask

How do I get tags in Selenium?

To get the tag name of a web element programmatically using Selenium in Java, use WebElement. getTagName() function. We first find the web element by name, id, class name, etc., and then call the getTagName() function on this web element.

Can I scrape using Selenium?

It extends its support to various browsers like Chrome, Internet Explorer, Safari, Edge, Firefox. To scrape data from these browsers, selenium provides a module called WebDriver, which is useful to perform various tasks like automated testing, getting cookies, getting screenshots, and many more.


1 Answers

If I look at the page source, for example in Chrome view-source:https://play.google.com/store/apps/details?id=com.teslacoilsw.launcher&hl=en. I also don't find a <div> element with attribute @itemprop and value price.

So your XPath is completely wrong. Also browser.find_element_by_xpath() returns an element and you want to extract the attribute value of @content. You should then use next:

priceValue = browser.find_element_by_xpath("//meta[@itemprop='price']")
print priceValue.get_attribute("content")
like image 162
Mark Veenstra Avatar answered Nov 13 '22 00:11

Mark Veenstra