Menu
How do I parse a table within a table for a simple web scraping application?
My process is:
HTML code:
<table border="0" cellpadding="2" cellspacing="1" width="660">
<tbody><tr bgcolor="#CCCCCC">
<th class="IntWxHeader">Time</th>
<th class="IntWxHeader">Weather</th>
<th class="IntWxHeader">Temperature</th>
<th class="IntWxHeader">Dewpoint</th>
<th class="IntWxHeader">Humidity</th>
<th class="IntWxHeader">Pressure</th>
<th class="IntWxHeader">Winds</th>
<th class="IntWxHeader">Visibility</th>
</tr>
<tr style="line-height:20pt; background-color:#E6EFFF;"><td align="center"><span style="font-size: 16px;">12:11AM</span></td><td align="center"><span style="font-size: 16px;">Cloudy</span></td><td align="center"><span style="font-size: 16px;">25°F/-4°C</span></td><td align="center"><span style="font-size: 16px;">25°F/-4°C</span></td><td align="center"><span style="font-size: 16px;">100%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">SSE 7 MPH</span></td><td align="center"><span style="font-size: 16px;">7mi.</span></td></tr><tr style="line-height:20pt; background-color:#F8FBFF;"><td align="center"><span style="font-size: 16px;">12:55AM</span></td><td align="center"><span style="font-size: 16px;">Mostly Cloudy</span></td><td align="center"><span style="font-size: 16px;">23°F/-5°C</span></td><td align="center"><span style="font-size: 16px;">23°F/-5°C</span></td><td align="center"><span style="font-size: 16px;">100%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">SSE 5 MPH</span></td><td align="center"><span style="font-size: 16px;">7mi.</span></td></tr><tr style="line-height:20pt; background-color:#E6EFFF;"><td align="center"><span style="font-size: 16px;">1:55AM</span></td><td align="center"><span style="font-size: 16px;">Partly Cloudy</span></td><td align="center"><span style="font-size: 16px;">25°F/-4°C</span></td><td align="center"><span style="font-size: 16px;">23°F/-5°C</span></td><td align="center"><span style="font-size: 16px;">93%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">S 5 MPH</span></td><td align="center"><span style="font-size: 16px;">7mi.</span></td></tr><tr style="line-height:20pt; background-color:#F8FBFF;"><td align="center"><span style="font-size: 16px;">2:55AM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">19°F/-7°C</span></td><td align="center"><span style="font-size: 16px;">19°F/-7°C</span></td><td align="center"><span style="font-size: 16px;">100%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">SSE 7 MPH</span></td><td align="center"><span style="font-size: 16px;">7mi.</span></td></tr><tr style="line-height:20pt; background-color:#E6EFFF;"><td align="center"><span style="font-size: 16px;">3:55AM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">23°F/-5°C</span></td><td align="center"><span style="font-size: 16px;">19°F/-7°C</span></td><td align="center"><span style="font-size: 16px;">86%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">SSW 6 MPH</span></td><td align="center"><span style="font-size: 16px;">7mi.</span></td></tr><tr style="line-height:20pt; background-color:#F8FBFF;"><td align="center"><span style="font-size: 16px;">4:55AM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">23°F/-5°C</span></td><td align="center"><span style="font-size: 16px;">19°F/-7°C</span></td><td align="center"><span style="font-size: 16px;">86%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">SSW 8 MPH</span></td><td align="center"><span style="font-size: 16px;">7mi.</span></td></tr><tr style="line-height:20pt; background-color:#E6EFFF;"><td align="center"><span style="font-size: 16px;">5:55AM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">23°F/-5°C</span></td><td align="center"><span style="font-size: 16px;">18°F/-8°C</span></td><td align="center"><span style="font-size: 16px;">80%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">WSW 3 MPH</span></td><td align="center"><span style="font-size: 16px;">7mi.</span></td></tr><tr style="line-height:20pt; background-color:#F8FBFF;"><td align="center"><span style="font-size: 16px;">6:07AM</span></td><td align="center"><span style="font-size: 16px;">Freezing Fog</span></td><td align="center"><span style="font-size: 16px;">21°F/-6°C</span></td><td align="center"><span style="font-size: 16px;">18°F/-8°C</span></td><td align="center"><span style="font-size: 16px;">86%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">NW 3 MPH</span></td><td align="center"><span style="font-size: 16px;">0.5mi.</span></td></tr><tr style="line-height:20pt; background-color:#E6EFFF;"><td align="center"><span style="font-size: 16px;">6:29AM</span></td><td align="center"><span style="font-size: 16px;">Freezing Fog</span></td><td align="center"><span style="font-size: 16px;">16°F/-9°C</span></td><td align="center"><span style="font-size: 16px;">16°F/-9°C</span></td><td align="center"><span style="font-size: 16px;">100%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">NW 3 MPH</span></td><td align="center"><span style="font-size: 16px;">0.25mi.</span></td></tr><tr style="line-height:20pt; background-color:#F8FBFF;"><td align="center"><span style="font-size: 16px;">6:55AM</span></td><td align="center"><span style="font-size: 16px;">Freezing Fog</span></td><td align="center"><span style="font-size: 16px;">16°F/-9°C</span></td><td align="center"><span style="font-size: 16px;">16°F/-9°C</span></td><td align="center"><span style="font-size: 16px;">100%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">WNW 3 MPH</span></td><td align="center"><span style="font-size: 16px;">0.25mi.</span></td></tr><tr style="line-height:20pt; background-color:#E6EFFF;"><td align="center"><span style="font-size: 16px;">7:55AM</span></td><td align="center"><span style="font-size: 16px;">Mist and Fog</span></td><td align="center"><span style="font-size: 16px;">18°F/-8°C</span></td><td align="center"><span style="font-size: 16px;">16°F/-9°C</span></td><td align="center"><span style="font-size: 16px;">93%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">SE 3 MPH</span></td><td align="center"><span style="font-size: 16px;">mi.</span></td></tr><tr style="line-height:20pt; background-color:#F8FBFF;"><td align="center"><span style="font-size: 16px;">8:09AM</span></td><td align="center"><span style="font-size: 16px;">Mist and Fog</span></td><td align="center"><span style="font-size: 16px;">18°F/-8°C</span></td><td align="center"><span style="font-size: 16px;">16°F/-9°C</span></td><td align="center"><span style="font-size: 16px;">93%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">ESE 5 MPH</span></td><td align="center"><span style="font-size: 16px;">5mi.</span></td></tr><tr style="line-height:20pt; background-color:#E6EFFF;"><td align="center"><span style="font-size: 16px;">8:55AM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">25°F/-4°C</span></td><td align="center"><span style="font-size: 16px;">23°F/-5°C</span></td><td align="center"><span style="font-size: 16px;">93%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">SSW 9 MPH</span></td><td align="center"><span style="font-size: 16px;">7mi.</span></td></tr><tr style="line-height:20pt; background-color:#F8FBFF;"><td align="center"><span style="font-size: 16px;">9:55AM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">36°F/2°C</span></td><td align="center"><span style="font-size: 16px;">27°F/-3°C</span></td><td align="center"><span style="font-size: 16px;">70%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">SSW 6 MPH</span></td><td align="center"><span style="font-size: 16px;">25mi.</span></td></tr><tr style="line-height:20pt; background-color:#E6EFFF;"><td align="center"><span style="font-size: 16px;">10:55AM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">43°F/6°C</span></td><td align="center"><span style="font-size: 16px;">23°F/-5°C</span></td><td align="center"><span style="font-size: 16px;">46%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">SSW 10 MPH</span></td><td align="center"><span style="font-size: 16px;">70mi.</span></td></tr><tr style="line-height:20pt; background-color:#F8FBFF;"><td align="center"><span style="font-size: 16px;">11:55AM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">48°F/9°C</span></td><td align="center"><span style="font-size: 16px;">21°F/-6°C</span></td><td align="center"><span style="font-size: 16px;">34%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">SSW 9 MPH</span></td><td align="center"><span style="font-size: 16px;">70mi.</span></td></tr><tr style="line-height:20pt; background-color:#E6EFFF;"><td align="center"><span style="font-size: 16px;">12:55PM</span></td><td align="center"><span style="font-size: 16px;">Partly Cloudy</span></td><td align="center"><span style="font-size: 16px;">52°F/11°C</span></td><td align="center"><span style="font-size: 16px;">9°F/-13°C</span></td><td align="center"><span style="font-size: 16px;">17%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">SW 8 MPH</span></td><td align="center"><span style="font-size: 16px;">70mi.</span></td></tr><tr style="line-height:20pt; background-color:#F8FBFF;"><td align="center"><span style="font-size: 16px;">1:55PM</span></td><td align="center"><span style="font-size: 16px;">Partly Cloudy</span></td><td align="center"><span style="font-size: 16px;">54°F/12°C</span></td><td align="center"><span style="font-size: 16px;">5°F/-15°C</span></td><td align="center"><span style="font-size: 16px;">14%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">N 3 MPH</span></td><td align="center"><span style="font-size: 16px;">70mi.</span></td></tr><tr style="line-height:20pt; background-color:#E6EFFF;"><td align="center"><span style="font-size: 16px;">2:55PM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">57°F/14°C</span></td><td align="center"><span style="font-size: 16px;">7°F/-14°C</span></td><td align="center"><span style="font-size: 16px;">13%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">WNW 6 MPH</span></td><td align="center"><span style="font-size: 16px;">70mi.</span></td></tr><tr style="line-height:20pt; background-color:#F8FBFF;"><td align="center"><span style="font-size: 16px;">3:55PM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">57°F/14°C</span></td><td align="center"><span style="font-size: 16px;">3°F/-16°C</span></td><td align="center"><span style="font-size: 16px;">11%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">NNW 7 MPH</span></td><td align="center"><span style="font-size: 16px;">70mi.</span></td></tr><tr style="line-height:20pt; background-color:#E6EFFF;"><td align="center"><span style="font-size: 16px;">4:55PM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">59°F/15°C</span></td><td align="center"><span style="font-size: 16px;">3°F/-16°C</span></td><td align="center"><span style="font-size: 16px;">10%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">NW 12 MPH</span></td><td align="center"><span style="font-size: 16px;">70mi.</span></td></tr><tr style="line-height:20pt; background-color:#F8FBFF;"><td align="center"><span style="font-size: 16px;">5:55PM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">57°F/14°C</span></td><td align="center"><span style="font-size: 16px;">-6°F/-21°C</span></td><td align="center"><span style="font-size: 16px;">7%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">W 20 MPH</span></td><td align="center"><span style="font-size: 16px;">70mi.</span></td></tr><tr style="line-height:20pt; background-color:#E6EFFF;"><td align="center"><span style="font-size: 16px;">6:55PM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">54°F/12°C</span></td><td align="center"><span style="font-size: 16px;">-2°F/-19°C</span></td><td align="center"><span style="font-size: 16px;">10%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">WNW 15 MPH</span></td><td align="center"><span style="font-size: 16px;">70mi.</span></td></tr><tr style="line-height:20pt; background-color:#F8FBFF;"><td align="center"><span style="font-size: 16px;">7:55PM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">46°F/8°C</span></td><td align="center"><span style="font-size: 16px;">5°F/-15°C</span></td><td align="center"><span style="font-size: 16px;">18%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">NW 7 MPH</span></td><td align="center"><span style="font-size: 16px;">70mi.</span></td></tr><tr style="line-height:20pt; background-color:#E6EFFF;"><td align="center"><span style="font-size: 16px;">8:55PM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">45°F/7°C</span></td><td align="center"><span style="font-size: 16px;">9°F/-13°C</span></td><td align="center"><span style="font-size: 16px;">23%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">W 7 MPH</span></td><td align="center"><span style="font-size: 16px;">25mi.</span></td></tr><tr style="line-height:20pt; background-color:#F8FBFF;"><td align="center"><span style="font-size: 16px;">9:55PM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">41°F/5°C</span></td><td align="center"><span style="font-size: 16px;">12°F/-11°C</span></td><td align="center"><span style="font-size: 16px;">31%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">NW 9 MPH</span></td><td align="center"><span style="font-size: 16px;">25mi.</span></td></tr><tr style="line-height:20pt; background-color:#E6EFFF;"><td align="center"><span style="font-size: 16px;">10:55PM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">43°F/6°C</span></td><td align="center"><span style="font-size: 16px;">14°F/-10°C</span></td><td align="center"><span style="font-size: 16px;">31%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">WNW 10 MPH</span></td><td align="center"><span style="font-size: 16px;">25mi.</span></td></tr><tr style="line-height:20pt; background-color:#F8FBFF;"><td align="center"><span style="font-size: 16px;">11:55PM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">34°F/1°C</span></td><td align="center"><span style="font-size: 16px;">23°F/-5°C</span></td><td align="center"><span style="font-size: 16px;">65%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">N 21 MPH</span></td><td align="center"><span style="font-size: 16px;">25mi.</span></td></tr></tbody></table>
I want to grab all of the data in the last line (
Here is what I have so far:
try:
resp = urlopen(url)
except URLError as e:
print "An error occured fetching %s \n %s" % (url, e.reason)
return 1
soup = BeautifulSoup(resp.read(), 'lxml')
# get outer table
try:
outerTables = soup.findAll('table')
except AttributeError as e:
print "No tables found, exciting"
return 1
# get inner table
try:
for table in outerTables:
innerTables = soup.findAll('table')
except AttributeError as e:
print "No inner tables found"
return 1
# get rows
try:
for table in innerTables:
rows = soup.findAll('tr')
except AttributeError as e:
print "No rows found"
return 1
# get headers
headers = []
try:
for row in rows:
markup = soup.findAll('th')
except AttributeError as e:
print "No headers found"
return 1
for i in range(0,len(markup)):
headers.append(markup[i].string)
Am I on the right path here?
Your html has one table with headers which are the th tags, some rows which are the tr tags and inside the rows is the data you want inside the td tags:
h = """<table border="0" cellpadding="2" cellspacing="1" width="660">
<tr bgcolor="#CCCCCC">
<th class="IntWxHeader">Time</th>
<th class="IntWxHeader">Weather</th>
<th class="IntWxHeader">Temperature</th>
<th class="IntWxHeader">Dewpoint</th>
<th class="IntWxHeader">Humidity</th>
<th class="IntWxHeader">Pressure</th>
<th class="IntWxHeader">Winds</th>
<th class="IntWxHeader">Visibility</th>
</tr>
<tr style="line-height:20pt; background-color:#E6EFFF;"><td align="center"><span style="font-size: 16px;">12:11AM</span></td><td align="center"><span style="font-size: 16px;">Cloudy</span></td><td align="center"><span style="font-size: 16px;">25°F/-4°C</span></td><td align="center"><span style="font-size: 16px;">25°F/-4°C</span></td><td align="center"><span style="font-size: 16px;">100%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">SSE 7 MPH</span></td><td align="center"><span style="font-size: 16px;">7mi.</span></td></tr><tr style="line-height:20pt; background-color:#F8FBFF;"><td align="center"><span style="font-size: 16px;">12:55AM</span></td><td align="center"><span style="font-size: 16px;">Mostly Cloudy</span></td><td align="center"><span style="font-size: 16px;">23°F/-5°C</span></td><td align="center"><span style="font-size: 16px;">23°F/-5°C</span></td><td align="center"><span style="font-size: 16px;">100%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">SSE 5 MPH</span></td><td align="center"><span style="font-size: 16px;">7mi.</span></td></tr><tr style="line-height:20pt; background-color:#E6EFFF;"><td align="center"><span style="font-size: 16px;">1:55AM</span></td><td align="center"><span style="font-size: 16px;">Partly Cloudy</span></td><td align="center"><span style="font-size: 16px;">25°F/-4°C</span></td><td align="center"><span style="font-size: 16px;">23°F/-5°C</span></td><td align="center"><span style="font-size: 16px;">93%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">S 5 MPH</span></td><td align="center"><span style="font-size: 16px;">7mi.</span></td></tr><tr style="line-height:20pt; background-color:#F8FBFF;"><td align="center"><span style="font-size: 16px;">2:55AM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">19°F/-7°C</span></td><td align="center"><span style="font-size: 16px;">19°F/-7°C</span></td><td align="center"><span style="font-size: 16px;">100%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">SSE 7 MPH</span></td><td align="center"><span style="font-size: 16px;">7mi.</span></td></tr><tr style="line-height:20pt; background-color:#E6EFFF;"><td align="center"><span style="font-size: 16px;">3:55AM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">23°F/-5°C</span></td><td align="center"><span style="font-size: 16px;">19°F/-7°C</span></td><td align="center"><span style="font-size: 16px;">86%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">SSW 6 MPH</span></td><td align="center"><span style="font-size: 16px;">7mi.</span></td></tr><tr style="line-height:20pt; background-color:#F8FBFF;"><td align="center"><span style="font-size: 16px;">4:55AM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">23°F/-5°C</span></td><td align="center"><span style="font-size: 16px;">19°F/-7°C</span></td><td align="center"><span style="font-size: 16px;">86%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">SSW 8 MPH</span></td><td align="center"><span style="font-size: 16px;">7mi.</span></td></tr><tr style="line-height:20pt; background-color:#E6EFFF;"><td align="center"><span style="font-size: 16px;">5:55AM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">23°F/-5°C</span></td><td align="center"><span style="font-size: 16px;">18°F/-8°C</span></td><td align="center"><span style="font-size: 16px;">80%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">WSW 3 MPH</span></td><td align="center"><span style="font-size: 16px;">7mi.</span></td></tr><tr style="line-height:20pt; background-color:#F8FBFF;"><td align="center"><span style="font-size: 16px;">6:07AM</span></td><td align="center"><span style="font-size: 16px;">Freezing Fog</span></td><td align="center"><span style="font-size: 16px;">21°F/-6°C</span></td><td align="center"><span style="font-size: 16px;">18°F/-8°C</span></td><td align="center"><span style="font-size: 16px;">86%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">NW 3 MPH</span></td><td align="center"><span style="font-size: 16px;">0.5mi.</span></td></tr><tr style="line-height:20pt; background-color:#E6EFFF;"><td align="center"><span style="font-size: 16px;">6:29AM</span></td><td align="center"><span style="font-size: 16px;">Freezing Fog</span></td><td align="center"><span style="font-size: 16px;">16°F/-9°C</span></td><td align="center"><span style="font-size: 16px;">16°F/-9°C</span></td><td align="center"><span style="font-size: 16px;">100%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">NW 3 MPH</span></td><td align="center"><span style="font-size: 16px;">0.25mi.</span></td></tr><tr style="line-height:20pt; background-color:#F8FBFF;"><td align="center"><span style="font-size: 16px;">6:55AM</span></td><td align="center"><span style="font-size: 16px;">Freezing Fog</span></td><td align="center"><span style="font-size: 16px;">16°F/-9°C</span></td><td align="center"><span style="font-size: 16px;">16°F/-9°C</span></td><td align="center"><span style="font-size: 16px;">100%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">WNW 3 MPH</span></td><td align="center"><span style="font-size: 16px;">0.25mi.</span></td></tr><tr style="line-height:20pt; background-color:#E6EFFF;"><td align="center"><span style="font-size: 16px;">7:55AM</span></td><td align="center"><span style="font-size: 16px;">Mist and Fog</span></td><td align="center"><span style="font-size: 16px;">18°F/-8°C</span></td><td align="center"><span style="font-size: 16px;">16°F/-9°C</span></td><td align="center"><span style="font-size: 16px;">93%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">SE 3 MPH</span></td><td align="center"><span style="font-size: 16px;">mi.</span></td></tr><tr style="line-height:20pt; background-color:#F8FBFF;"><td align="center"><span style="font-size: 16px;">8:09AM</span></td><td align="center"><span style="font-size: 16px;">Mist and Fog</span></td><td align="center"><span style="font-size: 16px;">18°F/-8°C</span></td><td align="center"><span style="font-size: 16px;">16°F/-9°C</span></td><td align="center"><span style="font-size: 16px;">93%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">ESE 5 MPH</span></td><td align="center"><span style="font-size: 16px;">5mi.</span></td></tr><tr style="line-height:20pt; background-color:#E6EFFF;"><td align="center"><span style="font-size: 16px;">8:55AM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">25°F/-4°C</span></td><td align="center"><span style="font-size: 16px;">23°F/-5°C</span></td><td align="center"><span style="font-size: 16px;">93%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">SSW 9 MPH</span></td><td align="center"><span style="font-size: 16px;">7mi.</span></td></tr><tr style="line-height:20pt; background-color:#F8FBFF;"><td align="center"><span style="font-size: 16px;">9:55AM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">36°F/2°C</span></td><td align="center"><span style="font-size: 16px;">27°F/-3°C</span></td><td align="center"><span style="font-size: 16px;">70%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">SSW 6 MPH</span></td><td align="center"><span style="font-size: 16px;">25mi.</span></td></tr><tr style="line-height:20pt; background-color:#E6EFFF;"><td align="center"><span style="font-size: 16px;">10:55AM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">43°F/6°C</span></td><td align="center"><span style="font-size: 16px;">23°F/-5°C</span></td><td align="center"><span style="font-size: 16px;">46%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">SSW 10 MPH</span></td><td align="center"><span style="font-size: 16px;">70mi.</span></td></tr><tr style="line-height:20pt; background-color:#F8FBFF;"><td align="center"><span style="font-size: 16px;">11:55AM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">48°F/9°C</span></td><td align="center"><span style="font-size: 16px;">21°F/-6°C</span></td><td align="center"><span style="font-size: 16px;">34%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">SSW 9 MPH</span></td><td align="center"><span style="font-size: 16px;">70mi.</span></td></tr><tr style="line-height:20pt; background-color:#E6EFFF;"><td align="center"><span style="font-size: 16px;">12:55PM</span></td><td align="center"><span style="font-size: 16px;">Partly Cloudy</span></td><td align="center"><span style="font-size: 16px;">52°F/11°C</span></td><td align="center"><span style="font-size: 16px;">9°F/-13°C</span></td><td align="center"><span style="font-size: 16px;">17%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">SW 8 MPH</span></td><td align="center"><span style="font-size: 16px;">70mi.</span></td></tr><tr style="line-height:20pt; background-color:#F8FBFF;"><td align="center"><span style="font-size: 16px;">1:55PM</span></td><td align="center"><span style="font-size: 16px;">Partly Cloudy</span></td><td align="center"><span style="font-size: 16px;">54°F/12°C</span></td><td align="center"><span style="font-size: 16px;">5°F/-15°C</span></td><td align="center"><span style="font-size: 16px;">14%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">N 3 MPH</span></td><td align="center"><span style="font-size: 16px;">70mi.</span></td></tr><tr style="line-height:20pt; background-color:#E6EFFF;"><td align="center"><span style="font-size: 16px;">2:55PM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">57°F/14°C</span></td><td align="center"><span style="font-size: 16px;">7°F/-14°C</span></td><td align="center"><span style="font-size: 16px;">13%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">WNW 6 MPH</span></td><td align="center"><span style="font-size: 16px;">70mi.</span></td></tr><tr style="line-height:20pt; background-color:#F8FBFF;"><td align="center"><span style="font-size: 16px;">3:55PM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">57°F/14°C</span></td><td align="center"><span style="font-size: 16px;">3°F/-16°C</span></td><td align="center"><span style="font-size: 16px;">11%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">NNW 7 MPH</span></td><td align="center"><span style="font-size: 16px;">70mi.</span></td></tr><tr style="line-height:20pt; background-color:#E6EFFF;"><td align="center"><span style="font-size: 16px;">4:55PM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">59°F/15°C</span></td><td align="center"><span style="font-size: 16px;">3°F/-16°C</span></td><td align="center"><span style="font-size: 16px;">10%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">NW 12 MPH</span></td><td align="center"><span style="font-size: 16px;">70mi.</span></td></tr><tr style="line-height:20pt; background-color:#F8FBFF;"><td align="center"><span style="font-size: 16px;">5:55PM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">57°F/14°C</span></td><td align="center"><span style="font-size: 16px;">-6°F/-21°C</span></td><td align="center"><span style="font-size: 16px;">7%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">W 20 MPH</span></td><td align="center"><span style="font-size: 16px;">70mi.</span></td></tr><tr style="line-height:20pt; background-color:#E6EFFF;"><td align="center"><span style="font-size: 16px;">6:55PM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">54°F/12°C</span></td><td align="center"><span style="font-size: 16px;">-2°F/-19°C</span></td><td align="center"><span style="font-size: 16px;">10%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">WNW 15 MPH</span></td><td align="center"><span style="font-size: 16px;">70mi.</span></td></tr><tr style="line-height:20pt; background-color:#F8FBFF;"><td align="center"><span style="font-size: 16px;">7:55PM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">46°F/8°C</span></td><td align="center"><span style="font-size: 16px;">5°F/-15°C</span></td><td align="center"><span style="font-size: 16px;">18%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">NW 7 MPH</span></td><td align="center"><span style="font-size: 16px;">70mi.</span></td></tr><tr style="line-height:20pt; background-color:#E6EFFF;"><td align="center"><span style="font-size: 16px;">8:55PM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">45°F/7°C</span></td><td align="center"><span style="font-size: 16px;">9°F/-13°C</span></td><td align="center"><span style="font-size: 16px;">23%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">W 7 MPH</span></td><td align="center"><span style="font-size: 16px;">25mi.</span></td></tr><tr style="line-height:20pt; background-color:#F8FBFF;"><td align="center"><span style="font-size: 16px;">9:55PM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">41°F/5°C</span></td><td align="center"><span style="font-size: 16px;">12°F/-11°C</span></td><td align="center"><span style="font-size: 16px;">31%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">NW 9 MPH</span></td><td align="center"><span style="font-size: 16px;">25mi.</span></td></tr><tr style="line-height:20pt; background-color:#E6EFFF;"><td align="center"><span style="font-size: 16px;">10:55PM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">43°F/6°C</span></td><td align="center"><span style="font-size: 16px;">14°F/-10°C</span></td><td align="center"><span style="font-size: 16px;">31%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">WNW 10 MPH</span></td><td align="center"><span style="font-size: 16px;">25mi.</span></td></tr><tr style="line-height:20pt; background-color:#F8FBFF;"><td align="center"><span style="font-size: 16px;">11:55PM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">34°F/1°C</span></td><td align="center"><span style="font-size: 16px;">23°F/-5°C</span></td><td align="center"><span style="font-size: 16px;">65%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">N 21 MPH</span></td><td align="center"><span style="font-size: 16px;">25mi.</span></td></tr>
/table>"""
from bs4 import BeautifulSoup
soup = BeautifulSoup(h)
table = soup.find("table")
# get all the headers/columns
headers = [th.text for th in table.find_all("th")]
print(", ".join(headers))
# select all rows in the table, after the first as the first includes the th tags which we already have.
rows = table.select("tr + tr")
# iterate over all the trs/rows
for row in rows:
# pull the text from each td
print(", ".join([td.text for td in row.find_all("td"))
Which would give you:
Time, Weather, Temperature, Dewpoint, Humidity, Pressure, Winds, Visibility
12:11AM, Cloudy, 25°F/-4°C, 25°F/-4°C, 100%, 0.00, SSE 7 MPH, 7mi.
12:55AM, Mostly Cloudy, 23°F/-5°C, 23°F/-5°C, 100%, 0.00, SSE 5 MPH, 7mi.
1:55AM, Partly Cloudy, 25°F/-4°C, 23°F/-5°C, 93%, 0.00, S 5 MPH, 7mi.
2:55AM, Mostly Clear, 19°F/-7°C, 19°F/-7°C, 100%, 0.00, SSE 7 MPH, 7mi.
3:55AM, Mostly Clear, 23°F/-5°C, 19°F/-7°C, 86%, 0.00, SSW 6 MPH, 7mi.
4:55AM, Mostly Clear, 23°F/-5°C, 19°F/-7°C, 86%, 0.00, SSW 8 MPH, 7mi.
5:55AM, Mostly Clear, 23°F/-5°C, 18°F/-8°C, 80%, 0.00, WSW 3 MPH, 7mi.
6:07AM, Freezing Fog, 21°F/-6°C, 18°F/-8°C, 86%, 0.00, NW 3 MPH, 0.5mi.
6:29AM, Freezing Fog, 16°F/-9°C, 16°F/-9°C, 100%, 0.00, NW 3 MPH, 0.25mi.
6:55AM, Freezing Fog, 16°F/-9°C, 16°F/-9°C, 100%, 0.00, WNW 3 MPH, 0.25mi.
7:55AM, Mist and Fog, 18°F/-8°C, 16°F/-9°C, 93%, 0.00, SE 3 MPH, mi.
8:09AM, Mist and Fog, 18°F/-8°C, 16°F/-9°C, 93%, 0.00, ESE 5 MPH, 5mi.
8:55AM, Mostly Clear, 25°F/-4°C, 23°F/-5°C, 93%, 0.00, SSW 9 MPH, 7mi.
9:55AM, Mostly Clear, 36°F/2°C, 27°F/-3°C, 70%, 0.00, SSW 6 MPH, 25mi.
10:55AM, Mostly Clear, 43°F/6°C, 23°F/-5°C, 46%, 0.00, SSW 10 MPH, 70mi.
11:55AM, Mostly Clear, 48°F/9°C, 21°F/-6°C, 34%, 0.00, SSW 9 MPH, 70mi.
12:55PM, Partly Cloudy, 52°F/11°C, 9°F/-13°C, 17%, 0.00, SW 8 MPH, 70mi.
1:55PM, Partly Cloudy, 54°F/12°C, 5°F/-15°C, 14%, 0.00, N 3 MPH, 70mi.
2:55PM, Mostly Clear, 57°F/14°C, 7°F/-14°C, 13%, 0.00, WNW 6 MPH, 70mi.
3:55PM, Mostly Clear, 57°F/14°C, 3°F/-16°C, 11%, 0.00, NNW 7 MPH, 70mi.
4:55PM, Mostly Clear, 59°F/15°C, 3°F/-16°C, 10%, 0.00, NW 12 MPH, 70mi.
5:55PM, Mostly Clear, 57°F/14°C, -6°F/-21°C, 7%, 0.00, W 20 MPH, 70mi.
6:55PM, Mostly Clear, 54°F/12°C, -2°F/-19°C, 10%, 0.00, WNW 15 MPH, 70mi.
7:55PM, Mostly Clear, 46°F/8°C, 5°F/-15°C, 18%, 0.00, NW 7 MPH, 70mi.
8:55PM, Mostly Clear, 45°F/7°C, 9°F/-13°C, 23%, 0.00, W 7 MPH, 25mi.
9:55PM, Mostly Clear, 41°F/5°C, 12°F/-11°C, 31%, 0.00, NW 9 MPH, 25mi.
10:55PM, Mostly Clear, 43°F/6°C, 14°F/-10°C, 31%, 0.00, WNW 10 MPH, 2
If you did have a table inside a table you would simply find the first then use first_table.find("table"), or if the table had a class or id you could maybe just use that to select the table directly.
106
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
