Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

BeautifulSoup findAll() given multiple classes?

I would like to scrape a list of items from a website, and preserve the order that they are presented in. These items are organized in a table, but they can be one of two different classes (in random order).

Is there any way to provide multiple classes and have BeautifulSoup4 find all items which are in any of the given classes?

I need to achieve what this code does, except preserve the order of items as it was in the source code:

items = soup.findAll(True,{'class':'class1'}) items += soup.findAll(True,{'class':'class2'}) 
like image 841
sebo Avatar asked Sep 10 '13 17:09

sebo


People also ask

What does BeautifulSoup findAll return?

find_all is used for returning all the matches after scanning the entire document. It is used for getting merely the first tag of the incoming HTML object for which condition is satisfied. It is used for getting all the incoming HTML objects for which condition is satisfied. The return type of find is <class 'bs4.

How do you get a href value in BeautifulSoup?

To get href with Python BeautifulSoup, we can use the find_all method. to create soup object with BeautifulSoup class called with the html string. Then we find the a elements with the href attribute returned by calling find_all with 'a' and href set to True .


2 Answers

you can do this

soup.findAll(True, {'class':['class1', 'class2']}) 

example:

>>> from bs4 import BeautifulSoup >>> soup = BeautifulSoup('<html><body><div class="class1"></div><div class="class2"></div><div class="class3"></div></body></html>') >>> soup.findAll(True, {"class":["class1", "class2"]}) [<div class="class1"></div>, <div class="class2"></div>] 
like image 108
Roman Pekar Avatar answered Sep 23 '22 06:09

Roman Pekar


I am new to Python with BeautifulSoup but may be my answer help you. I came across the same situation where I have to find multiple classes of one tag so, I just pass the classes into an array and it works for me. Here is the code snippet

# Search with single Class     find_all("tr",  {"class":"abc"}) # Search with multiple classes     find_all("tr",  {"class": ["abc", "xyz"]}) 
like image 26
Bhoopi Avatar answered Sep 21 '22 06:09

Bhoopi