How to create the scrapy project by python3

Question

I use Mac OS, it has python 2.7 and python 3.4. I use the pip install command to to install scrapy in python 2.7. Buy I also use the pip3 install command to install scrapy in python3.4 too...

I read the official documents on scrapy.org, I know that the scrapy just support the python 2.7. When I use the command scrapy startproject tutorial, it will return the errow below.

How can I use the command scrapy startproject tutorial with python 2.7?

  File "/Library/Frameworks/Python.framework/Versions/3.4/bin/scrapy", line 9, in <module>
    load_entry_point('Scrapy==1.1.0dev1', 'console_scripts', 'scrapy')()
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/Scrapy-1.1.0dev1-py3.4.egg/scrapy/cmdline.py", line 122, in execute
    cmds = _get_commands_dict(settings, inproject)
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/Scrapy-1.1.0dev1-py3.4.egg/scrapy/cmdline.py", line 46, in _get_commands_dict
    cmds = _get_commands_from_module('scrapy.commands', inproject)
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/Scrapy-1.1.0dev1-py3.4.egg/scrapy/cmdline.py", line 29, in _get_commands_from_module
    for cmd in _iter_command_classes(module):
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/Scrapy-1.1.0dev1-py3.4.egg/scrapy/cmdline.py", line 21, in _iter_command_classes
    for obj in vars(module).itervalues():
AttributeError: 'dict' object has no attribute 'itervalues'

Answer 1

The solution there is, the developer had already provide the scrapy for Python3.x, so you can try the

$ pip install scrapy==1.1.0rc1

And the article is here

Answer 2

Installing scrapy with pip will put an executable file somewhere in your PATH. Since you installed it two times, the python2 version was probably overwritten. To find this file use the command which scrapy. To see the content of the file use cat $(which scrapy). It probably contains a line the reads something like this: #!/usr/bin/python3.4 causing it to use an incompatible version of python.

To fix this, uninstall the python3 version of scrapy.

pip3 uninstall scrapy

Then clean the command cache in bash by using hash -r or starting a new terminal session.

If the scrapy command still doesn't work you might have to reinstall the python 2 version of it as well.

pip install scrapy --force-reinstall