Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Retrieving multiple tags via beautifulsoup css selector

Is there any way to provide multiple arguments to beautifulsoup's select method?

I'm retrieving data via soup.select('div[class^="TypeA"]'. This fetches me all the divs where classes match the pattern TypeA. I'm interested in retrieving, in addition, another div where class="TypeB" (exact match).

Now I can possibly do this in two separate passes, e.g. something like:

r = requests.get(jurl)
soup = BeautifulSoup(r.text,"lxml")
list1 = []
#get typeA divs
for div in soup.select('div[class^="TypeA"]'):
    t = [text for text in div.stripped_strings]
    list1.append(t)
list2 = []
#get typeB divs
for div in soup.select('div[class^="TypeB"]'):
    t = [text for text in div.stripped_strings]
    list2.append(t)
#combine the two into tuples. Both lists are of the same size
list3 = []
count = 0
for item in list1:
    list3.append((item,list2[count]))
    count += 1    
print list3

But is it possible to do it in a single pass? Going through the documentation, it's not immediately obvious how this can be done.

like image 452
Hassan Baig Avatar asked Feb 11 '17 15:02

Hassan Baig


1 Answers

soup.select('div[class^="TypeA"], div[class^="TypeB"]')

Use , to use multiple selectors

CSS Selector Reference

enter image description here

like image 109
宏杰李 Avatar answered Nov 15 '22 00:11

宏杰李