beautifulsoup can't find href in file using regular expression

Question

I have a html file like following:

<form action="/2811457/follow?gsid=3_5bce9b871484d3af90c89f37" method="post">
<div>
<a href="/2811457/follow?page=2&amp;gsid=3_5bce9b871484d3af90c89f37">next_page</a>
&nbsp;<input name="mp" type="hidden" value="3" />
<input type="text" name="page" size="2" style='-wap-input-format: "*N"' />
<input type="submit" value="jump" />&nbsp;1/3
</div>
</form>

how to extract the href ""/2811457/follow?page=2&gsid=3_5bce9b871484d3af90c89f37" in next_page?

It is a part of html,I intend to make it clear. When I use beautifulsoup,

print soup.find('a',href=re.compile('follow?page'))

it return None,why? I'm new to beautifulsoup,and I have look the document,but still confused.

now I use an ugly way:

    urls = soup.findAll('a',href=True))
    for url in urls:
        if follow?page in url:
            print url

I need a more clear and elegant way.

Answer 1

You need to escape the question mark. The regular expression w? means zero or one w. Try this:

print soup.find('a', href = re.compile(r'.*follow\?page.*'))