runspider: error: File not found: - Scrapy

Question

After working through the official tut I decided to try to build my own spider in the same project. I created parker_spider.py in spiders which contains:

start_urls = [
    "myurl"
]


class Parker_Spider(scrapy.Spider):

    name = "parker"


    def make_requests(self):
        for i in range(self.max_id):
            yield Request('myurl', method="post", headers= headers, body=payload, callback=self.parse_method)


    def parse_method(self,response):
        print(response.body) 

When I run:

$ scrapy runspider parker
2016-05-25 20:26:42 [scrapy] INFO: Scrapy 1.1.0 started (bot: tutorial)
2016-05-25 20:26:42 [scrapy] INFO: Overridden settings: {'NEWSPIDER_MODULE': 'tutorial.spiders', 'SPIDER_MODULES': ['tutorial.spiders'], 'ROBOTSTXT_OBEY': True, 'BOT_NAME': 'tutoria
l'}
Usage
=====
  scrapy runspider [options] <spider_file>

runspider: error: File not found: parker

what am I doing wrong?

Answer 1

The runspider command expects a spider filename, not the spider name:

$ scrapy runspider parker_spider.py

And, if you have created a Scrapy project and are running spiders from inside the project directory, better use the crawl command instead - here you should use a spider name:

$ scrapy crawl parker