urllib2 not returning full webpage

Question

I'm just starting out in Python and I'm trying to request the html source code of a site using urllib2. However when I try and get the html content from a site I'm not getting the full html content - there are tags missing. I know they're missing as when I view the site in firebug the code shows up. Is this due to the way I'm requesting the data - or due to the site? If so is there a way in which I can get the full source code of the site in python, and then parse it?

Currently the code I'm using to request the content and the site I'm trying is:

import urllib2

url = 'http://marinetraffic.com/ais/'
response = urllib2.urlopen(url)
html = response.read()
print(html)

Specifically the content between the - div id="map_area" - is missing. Any help/pointers greatly appreciated!

Answer 1

You are getting incomplete data because most of the content on this page is dynamically generated via Javascript...