Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

scrapy on Macbook error: Module 'tutorial' already exists

After installation of scrapy, I use 'scrapy startproject tutorial' to start but it shows below:

Icelesss-MacBook-Pro:tutorial iceless$ scrapy startproject tutorial
Error: Module 'tutorial' already exists

When I just type import scrapy, it shows:

Icelesss-MacBook-Pro:~ iceless$ import scrapy
-bash: import: command not found
Icelesss-MacBook-Pro:~ iceless$ scrapy.item
-bash: scrapy.item: command not found
Icelesss-MacBook-Pro:~ iceless$ create scrapy.Item
-bash: create: command not found
Icelesss-MacBook-Pro:~ iceless$ items.py
-bash: items.py: command not found
Icelesss-MacBook-Pro:~ iceless$ cd tutorial
Icelesss-MacBook-Pro:tutorial iceless$ import scrapy
-bash: import: command not found
like image 235
Bryan Xue Avatar asked Jul 25 '26 23:07

Bryan Xue


1 Answers

This should be a relatively simple fix. It seems on some platforms that when installing scrapy a symlink does not get created or the cli tool does not get added to your $PATH. The first thing that needs to happen is finding the location of your current Python and version:

$ which python
/opt/local/bin/python # your python location may be different

$ python -V
Python 3.6.5 # your version may be different (we need the first two digits)

Take the first two paths of the which python command and prepend it to:

/Library/Frameworks/Python.framework/Versions/3.6/bin/scrapy

So you end up with:

/opt/local/Library/Frameworks/Python.framework/Versions/3.6/bin/scrapy

Now the final step is to create an alias for the command in ~/.bash_profile:

alias scrapy="/opt/local/Library/Frameworks/Python.framework/Versions/3.6/bin/scrapy"

I've created a Bash script which should (hopefully) automate this process:

scrapy_alias.sh

#!/bin/bash

a=$(command -v python)
b=$(python -V | grep -oE "\\d\\.\\d")
c="${a%/bin*}/Library/Frameworks/Python.framework/Versions/${b}/bin/scrapy"

printf "\\n# Scrapy alias\\nalias scrapy=\"${c}\"\\n" | sudo tee -a ~/.bash_profile

In terminal run the script, then source the changes to ~/.bash_profile:

$ ./scrapy_alias.sh
$ . ~/.bash_profile

Now you should be able to start the tutorial:

$ scrapy startproject tutorial

New Scrapy project 'tutorial', using template directory '/opt/local/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/scrapy/templates/project', created in:
    /Users/Username/Desktop/Scrapy_Tutorial

You can start your first spider with:
    cd tutorial
    scrapy genspider example example.com
like image 119
l'L'l Avatar answered Jul 28 '26 12:07

l'L'l