I am trying to scrape a table website with mechanize. I want to scrape the second row.
When I run :
agent.page.search('table.ea').search('tr')[-2].search('td').map{ |n| n.text }
I would expect it to scrape the whole row. But instead it only scrapes: ["2011-02-17", "0,00"]
Why isn't it scraping all of the columns in the row, but just the first and the last column?
Xpath: /html/body/center/table/tbody/tr[2]/td[2]/table/tbody/tr[3]/td/table/tbody/tr[2]/td/table/tbody/tr[2]
CSS PATH: html body center table tbody tr td table tbody tr td table tbody tr td table.ea tbody tr td.total
The page is similar to this:
<table><table><table>
<table width="100%" border="0" cellpadding="0" cellspacing="1" class="ea">
<tr>
<th><a href="#">Date</a></th>
<th><a href="#">One</a></th>
<th><a href="#">Two</a></th>
<th><a href="#">Three</a></th>
<th><a href="#">Four</a></th>
<th><a href="#">Five</a></th>
<th><a href="#">Six</a></th>
<th><a href="#">Seven</a></th>
<th><a href="#">Eight</a></th>
</tr>
<tr>
<td><a href="#">2011-02-17</a></td>
<td align="right">0</td>
<td align="right">0</td>
<td align="right">0,00</td>
<td align="right">0</td>
<td align="right">0</td>
<td align="right">0</td>
<td align="right">0</td>
<td align="right">387</td>
<td align="right">0,00</td> <!-- FOV -->
<td align="right">0,00</td>
</tr>
<tr>
<td class="total">Ialt</td>
<td class="total" align="right">0</td>
<td class="total" align="right">40</td>
<td class="total" align="right">0,46</td>
<td class="total" align="right">2</td>
<td class="total" align="right">0</td>
<td class="total" align="right">0</td>
<td class="total" align="right">0</td>
<td class="total" align="right">3.060</td>
<td class="total" align="right">0,00</td>
<td class="total" align="right">18,58</td>
</tr>
</table>
</table></table></table>
Using the following Ruby code (https://gist.github.com/835603):
require 'mechanize'
require 'pp'
a = Mechanize.new { |agent|
agent.user_agent_alias = 'Mac Safari'
}
a.get('http://binarymuse.net/table.html') do |page|
pp page.search('table.ea').search('tr')[-2].search('td').map{ |n| n.text }
end
I get the following output:
["2011-02-17", "0", "0", "0,00", "0", "0", "0", "0", "387", "0,00", "0,00"]
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With