BeautifulSoup - easy way to to obtain HTML-free contents

Question

I'm using this code to find all interesting links in a page:

soup.findAll('a', href=re.compile('^notizia.php\?idn=\d+'))

And it does its job pretty well. Unfortunately inside that a tag there are a lot of nested tags, like font, b and different things... I'd like to get just the text content, without any other html tag.

Example of link:

<A HREF="notizia.php?idn=1134" OnMouseOver="verde();" OnMouseOut="blu();"><FONT CLASS="v12"><B>03-11-2009:&nbsp;&nbsp;<font color=green>CCS Ingegneria Elettronica-Sportello studenti ed orientamento</B></FONT></A>

Of course it's ugly (and the markup is not always the same!) and I'd like to get:

03-11-2009:  CCS Ingegneria Elettronica-Sportello studenti ed orientamento

In the documentation it says to use text=True in findAll method, but it will ignore my regex. Why? How can I solve that?

Answer 1

I've used this:

def textOf(soup):
    return u''.join(soup.findAll(text=True))

So...

texts = [textOf(n) for n in soup.findAll('a', href=re.compile('^notizia.php\?idn=\d+'))]