Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

python scrapy does not working - "ImportError: No module named settings"

Tags:

python

scrapy

scrappy lib in /usr/lib/python2.7/site-packages/scrapy
my project catalog:

.../projects/scrapy                                       
.../projects/parser_module                                     
....../projects/parser_module/parser  
....../projects/parser_module/parser           
........../projects/parser_module/parser/spiders/.....     
........../projects/parser_module/parser/<files etc>   
....../projects/parser_module/scrapy.cfg

In directory .../projects/parser_module/ I am set command scrapy crawl parser and get result:

Traceback (most recent call last):                                                                   

 File "/usr/bin/scrapy", line 4, in <module>                                                         
   execute()                                                                                         
  File "/usr/lib/python2.7/site-packages/scrapy/cmdline.py", line 109, in execute  
  settings = get_project_settings() 
  File "/usr/lib/python2.7/site-packages/scrapy/utils/project.py", line 60, in get_project_settings  
    settings.setmodule(settings_module_path, priority='project')          
  File "/usr/lib/python2.7/site-packages/scrapy/settings/__init__.py", line 108, in setmodule        
    module = import_module(module)                                                                   
  File "/usr/lib64/python2.7/importlib/__init__.py", line 37, in import_module                       
    __import__(name)                                                                                 
ImportError: No module named settings  

Can you please tell me how to solve the problem?

like image 896
Popov Dmytro Avatar asked Sep 30 '22 09:09

Popov Dmytro


2 Answers

to avoid such issue create your project folder with scrapy startproject parser_module

now to fix your issue, you either have a fresh start or create a dummy project with scrapy startproject to copy setting.py from. Then maybe next error you will find other missing file in this folder.

This is the typical structure of a scrapy project.

.
├── scrapy.cfg
└── project_name
    ├── __init__.py
    ├── items.py
    ├── settings.py
    └── spiders
        └── __init__.py
            spider.py
like image 121
Frederic Bazin Avatar answered Oct 11 '22 03:10

Frederic Bazin


To solve "ImportError: No module named *.settings", try to change project name and related settings in scrapy.cfg

like image 39
piawine Avatar answered Oct 11 '22 03:10

piawine