Scrape data from a table with scrapy

Question

Scrape data from a table with scrapy. The table html is like:

<table class="tablehd">

<tr class="colhead">
<td width="170">MON, NOV 11</td>
<td width="80">Item</td>
<td width="60" align="center"></td>
<td width="210">Item</td>
<td width="220">Item</td>
</tr>

<tr class="oddrow">
<td> Item </a></td>
<td> Item </td>
<td align="center"> Item </td>
<td></td>
<td> Item </td>
</tr>

<tr class="evenrow">
<td> Item </a></td>
<td> Item </td>
<td align="center"> Item </td>
<td></td>
<td> Item </td>
</tr>


</table>

Entire list is avialable by

items = hxs.select('//table[@class="tablehd"]//td//text()').extract()

How would you split them to each item and then assign data td1 - td5ta

Answer 1

Not sure what exactly do you want to see in your items, but here's an example and I hope this is it:

class MyItem(Item):
    value = Field()


class MySpider(BaseSpider):
    ...

    def parse(self, response):
        hxs = HtmlXPathSelector(response)
        items = hxs.select('//table[@class="tablehd"]/td')

        for item in items:
            my_item = MyItem()
            my_item['value'] = item.select('.//text()').extract()
            yield my_item

Hope that helps.