Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Scrapy does not have command 'crawl'

Tags:

python

scrapy

I started to learn Scrapy but right away I get an error Unknown command: crawl. I do not know why im getting this, but in py Scrapy commands I do not have that command. Im using python 3.6 and pycharm as editor.

(venv) C:\Users\Pc\PycharmProjects\web skreper\venv\Scripts>scrapy crawl quotes
Scrapy 1.7.3 - no active project

Unknown command: crawl

Use "scrapy" to see available commands

(venv) C:\Users\Pc\PycharmProjects\web skreper\venv\Scripts>scrapy
Scrapy 1.7.3 - no active project

Usage:
  scrapy <command> [options] [args]

Available commands:
  bench         Run quick benchmark test
  fetch         Fetch a URL using the Scrapy downloader
  genspider     Generate new spider using pre-defined templates
  runspider     Run a self-contained spider (without creating a project)
  settings      Get settings values
  shell         Interactive scraping console
  startproject  Create new project
  version       Print Scrapy version
  view          Open URL in browser, as seen by Scrapy

This is the basic code that I wrote:

import scrapy

class My_Scraper(scrapy.Spider):

    name = 'quotes'

    #url sajtova koje scrapujem
    start_urls = [
        'https://www.nekretnine.rs/'
        ]

    def parse(self, response):

        title = response.css('title').extract()

        yield {'title text': title}
like image 662
taga Avatar asked Aug 11 '26 21:08

taga


1 Answers

You need to be inside the project folder within the Scrapy folder.

You are currently trying to run the command from C:\Users\Pc\PycharmProjects\web skreper\venv\Scripts

but it should be something like C:\Users\Pc\PycharmProjects\web skreper\venv\Scripts\Scrapy\My_Scraper

like image 184
reg202 Avatar answered Aug 14 '26 11:08

reg202