Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Scrapy spider not found error

Tags:

python

scrapy

This is Windows 7 with python 2.7

I have a scrapy project in a directory called caps (this is where scrapy.cfg is)

My spider is located in caps\caps\spiders\campSpider.py

I cd into the scrapy project and try to run

scrapy crawl campSpider -o items.json -t json 

I get an error that the spider can't be found. The class name is campSpider

...     spider = self.crawler.spiders.create(spname, **opts.spargs)   File "c:\Python27\lib\site-packages\scrapy-0.14.0.2841-py2.7-win32.egg\scrapy\spidermanager.py", l ine 43, in create     raise KeyError("Spider not found: %s" % spider_name) KeyError: 'Spider not found: campSpider' 

Am I missing some configuration item?

like image 597
user199421 Avatar asked Mar 26 '12 17:03

user199421


1 Answers

Make sure you have set the "name" property of the spider. Example:

class campSpider(BaseSpider):    name = 'campSpider' 

Without the name property, the scrapy manager will not be able to find your spider.

like image 176
Sjaak Trekhaak Avatar answered Sep 29 '22 08:09

Sjaak Trekhaak