What's the best way to get a description of the website, in Python?

Question

Suppose I downloaded the HTML code, and I can parse it. How do I get the "best" description of that website, if that website does not have meta-description tag?

Answer 1

You could get the first few sentence returned from something like Readability.

Safari 5 uses it, so it must be alright :)

Answer 2

To follow up on the "Readability" suggestion above (which itself is inspired by the website InstaPaper), they have release the JavaScript: http://code.google.com/p/arc90labs-readability/. What's more, some guy took that and ported it to python: http://github.com/gfxmonk/python-readability. Rejoice!