Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

TypeError: 'float' object is not iterable i'm using scrapy and python 3.5

Tags:

python

scrapy

I have a problem with my scrapyspider which uses the following code:

class MySpider(BaseSpider):
    name = "craig"
    allowed_domains = ["craigslist.org"]
    start_urls = ["http://sfbay.craigslist.org/search/npo"]

    def parse(self, response):
    hxs = HtmlXPathSelector(response)
    titles = hxs.xpath("//span[@class='pl']")
    items = []
    for i in titles:
        item = CraigslistSampleItem()
        item["title"] = i.select("a/text()").extract()
        item["link"] = i.select("a/@href").extract()
        items.append(item)
    return items

This is my items.py

from scrapy.item import Item, Field

class CraigslistSampleItem(Item):
    title = Field()
    link = Field()

When I run the spider with scrapy crawl craig, I get the following error:

TypeError: 'float' object is not iterable
2017-03-07 10:13:44 [scrapy.core.scraper] ERROR: Error downloading <GET http://sfbay.craigslist.org/search/npo>
TypeError: 'float' object is not iterable

How can I fix this? Thanks

like image 454
Alfan Dinda Rahmawan Avatar asked Dec 19 '25 00:12

Alfan Dinda Rahmawan


1 Answers

I encountered a similar error after installing scrapy using conda which resulted in getting scrapy version 1.1 and twisted version 17.1.0.

I believe there is an incompatibility between scrapy versions less than 1.3.2 and twisted version 17.x. I was able to resolve it by downgrading twisted to version 16.x or upgrading scrapy to version 1.3.3.

like image 185
nicolaskruchten Avatar answered Dec 20 '25 18:12

nicolaskruchten