Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Change attrs within HTML tag to view full content Python BeautifulSoup

I am trying to view the full content of the website fortune.com/best-companies The original code has the following tag in its script :

<nav id="bottom-panel-pagination" class="panel-pagination hasNextOnly">
    <div data-event="view left" class="prev-page icon-new-left-arrow"></div>
    <div data-event="view right" class="next-page icon-new-right-arrow"></div>
</nav>

I want to change the class attribute "panel-pagination hasNextOnly" to "panel-pagination hasNoPagination" using BeautifulSoup. My python code looks like this :

import urllib2
from bs4 import BeautifulSoup
quote_page = "http://fortune.com/best-companies/"
page = urllib2.urlopen(quote_page)
soup = BeautifulSoup(page, "html.parser")
fullpage = soup.find('nav', attrs = {'class' : 'panel-pagination hasNextOnly'})
print fullpage

I want to change attrs = {'class' : 'panel-pagination hasNextOnly'} to attrs = {'class' : 'panel-pagination hasNoPagination'}

The website should reload after this so that I can further scrap it. How do I do it? Please help.

like image 636
user3615286 Avatar asked Dec 27 '16 18:12

user3615286


1 Answers

Beautifulsoup provides facility to change attributes as..

soup.find('nav')['attribute'] = 'new-attribute-values'


In your case.
fullpage = soup.find('nav', attrs = {'class' : 'panel-pagination hasNextOnly'})
fullpage['class'] = 'panel-pagination hasNoPagination'
print (fullpage)
like image 195
Jimish Fotariya Avatar answered Nov 01 '22 21:11

Jimish Fotariya