I have a problem with my scrapyspider which uses the following code:
class MySpider(BaseSpider):
name = "craig"
allowed_domains = ["craigslist.org"]
start_urls = ["http://sfbay.craigslist.org/search/npo"]
def parse(self, response):
hxs = HtmlXPathSelector(response)
titles = hxs.xpath("//span[@class='pl']")
items = []
for i in titles:
item = CraigslistSampleItem()
item["title"] = i.select("a/text()").extract()
item["link"] = i.select("a/@href").extract()
items.append(item)
return items
This is my items.py
from scrapy.item import Item, Field
class CraigslistSampleItem(Item):
title = Field()
link = Field()
When I run the spider with scrapy crawl craig, I get the following error:
TypeError: 'float' object is not iterable
2017-03-07 10:13:44 [scrapy.core.scraper] ERROR: Error downloading <GET http://sfbay.craigslist.org/search/npo>
TypeError: 'float' object is not iterable
How can I fix this? Thanks
I encountered a similar error after installing scrapy using conda which resulted in getting scrapy version 1.1 and twisted version 17.1.0.
I believe there is an incompatibility between scrapy versions less than 1.3.2 and twisted version 17.x. I was able to resolve it by downgrading twisted to version 16.x or upgrading scrapy to version 1.3.3.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With