Fetch all href link using selenium in python

Question

I am practicing Selenium in Python and I wanted to fetch all the links on a web page using Selenium.

For example, I want all the links in the href= property of all the <a> tags on http://psychoticelites.com/

I've written a script and it is working. But, it's giving me the object address. I've tried using the id tag to get the value, but, it doesn't work.

My current script:

from selenium import webdriver
from selenium.webdriver.common.keys import Keys


driver = webdriver.Firefox()
driver.get("http://psychoticelites.com/")

assert "Psychotic" in driver.title

continue_link = driver.find_element_by_tag_name('a')
elem = driver.find_elements_by_xpath("//*[@href]")
#x = str(continue_link)
#print(continue_link)
print(elem)

Answer 1

Well, you have to simply loop through the list:

elems = driver.find_elements_by_xpath("//a[@href]")
for elem in elems:
    print(elem.get_attribute("href"))

find_elements_by_* returns a list of elements (note the spelling of 'elements'). Loop through the list, take each element and fetch the required attribute value you want from it (in this case href).

Answer 2

I have checked and tested that there is a function named find_elements_by_tag_name() you can use. This example works fine for me.

elems = driver.find_elements_by_tag_name('a')
    for elem in elems:
        href = elem.get_attribute('href')
        if href is not None:
            print(href)

Answer 3

You can try something like:

    links = driver.find_elements_by_partial_link_text('')