Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Beautiful soup - capture all links with a certain class or text

I'm trying to capture all the relevant links from a web page with beautiful soup. All the links I need have both the class="btn btn-gray" and also the text <a...>More Info<>

What's the best way to extract just these links?

like image 552
Yunti Avatar asked Jan 08 '23 11:01

Yunti


1 Answers

How about this?

soup = BeautifulSoup(html, 'lxml')

all_links = []
links = soup.find_all('a', {'class': ['btn', 'btn-gray']})
for link in links:
    if 'More Info' in link.text:
        all_links.append(link['href'])  # Save href only, for example.

or as a clean list comprehension:

links = soup.find_all('a', {'class': ['btn', 'btn-gray']})
results = [link['href'] for link in links if 'More Info' in link.text]
like image 63
supermitch Avatar answered Jan 25 '23 04:01

supermitch