BeautifulSoup not extracting all html (automatically deleting much of a page's html)

Question

I am trying to use BeautifulSoup to extract the contents from a website (http://brooklynexposed.com/events/). As an example of the problem I can run the following code:

import urllib
import bs4 as BeautifulSoup

url = 'http://brooklynexposed.com/events/'
html = urllib.urlopen(url).read()

soup = BeautifulSoup.BeautifulSoup(html)
print soup.prettify().encode('utf-8')

The output seems to cut off the html as follows:

       <li class="event">
        9:00pm - 11:00pm
        <br/>
        <a href="http://brooklynexposed.com/events/entry/5432/2013-07-16">
         Comedy Sh
        </a>
       </li>
      </ul>
     </div>
    </div>
   </div>
  </div>
 </body>
</html>

It is cutting off the listing with the name Comedy Show along with all html that comes after until the final closing tags. Majority of the html is being automatically removed. I have noticed similar things on numerous website, that if the page is too long, BeautifulSoup fails to parse the entire page and just cuts out text. Does anyone have a solution for this? If BeautifulSoup is not capable of handling such pages, does anyone know any other libraries with functions similar to prettify()?

Answer 1

I had troubles that bs4 cuts html on some machines and on some not. It was not reproducable....

I switched to this:

soup = bs4.BeautifulSoup(html, 'html5lib')

.. and it works now.