How to remove all html tags from downloaded page

Question

I have downloaded a page using urlopen. How do I remove all html tags from it? Is there any regexp to replace all <*> tags?

Answer 1

You could use html2text which is supposed to make a readable text equivalent from an HTML source (programatically with Python or as a command-line tool). Thus I may extrapolate your needs from your question...

Answer 2

I can also recommend BeautifulSoup which is an easy to use html parser. There you would do something like:

from BeautifulSoup import BeautifulSoup

soup = BeautifulSoup(html)
all_text = ''.join(soup.findAll(text=True))

This way you get all the text from a html document.

Answer 3

There's a great python library called bleach. This call below will remove all html tags, leaving everything else (but not removing the content inside tags that are not visible).

bleach.clean(thestring, tags=[], attributes={}, styles=[], strip=True)