Determine the scrapy root directory

Question

I'd like to be able to read a file relative to the scrapy root directory, as described in the documentation.

Is there a way of determining this path from inside the spider?

Answer 1

I don't know if scrapy is aware of where it is located on the filesystem, but in my settings.py i add

PROJECT_ROOT = os.path.abspath(os.path.dirname(__file__))

then in spider you can say

from yourscrapyprojecthere import settings

settings.PROJECT_ROOT

Here I have assumed my project root is wherethe settings.py file is located. If you want to get the directory where scrapy.cfg is you have to go up a directory

Answer 2

Very old question but for those often looking for the answer, scrapy has a built-in way of finding the project root:

from scrapy.utils.conf import closest_scrapy_cfg

proj_root = closest_scrapy_cfg()

will return the absolute path of the folder containing the scrapy.cfg file

EDIT: will return the absolute path of the scrapy.cfg file for the project