Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Python: How to turn two for-statements into one?

I'm trying to parse a web table and export certain data into a csv file.

I'm ignorant to forming two XPaths followed by a single for-statement (or maybe two is correct?).

Current Spider:

class MySpider(BaseSpider):
    symbols = ["SCMP"]
    name =  "dozen"
    allowed_domains = ["yahoo.com"]     
    start_urls = ["http://finance.yahoo.com/q/is?s=SCMP&annual"]

    def parse(self, response):
        hxs = HtmlXPathSelector(response)
        revenue = response.xpath('//td[@align="right"]/strong/text()')
        date = response.xpath('//tr[@class="yfnc_modtitle1"]/th/text()')
        items = []
        for rev in revenue:
            item = DozenItem()
            item["Revenue"] = rev.re('\d*,\d*')
            items.append(item)
        return items[:3]
        days = []
        for day in dates:
            item = DozenItem()
            item["Date"] = day.re('\d*')
            days.append(item)
        return items[:3]

I know this needs work, I'm just not sure which direction to go...?

This is the output:

Export

As is visible, I can't get the dates to fill in.

Here is the html I'm parsing the dates from:

<TR class="yfnc_modtitle1" style="border-top:none;">

<th scope="col" style="border-top:2px solid #000;text-align:right; font-weight:bold">Dec 31, 2014</th>

<th scope="col" style="border-top:2px solid #000;text-align:right; font-weight:bold">Dec 31, 2013</th>
<th scope="col" style="border-top:2px solid #000;text-align:right; font-weight:bold">Dec 31, 2012</th>
</TR>
<tr>
<td colspan="2">
like image 985
Charles Watson Avatar asked Nov 29 '25 12:11

Charles Watson


1 Answers

for rev, day in zip(revenue, dates):
    pass # code here
like image 114
TigerhawkT3 Avatar answered Dec 02 '25 02:12

TigerhawkT3



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!