I'd like to be able to read a file relative to the scrapy root directory, as described in the documentation.
Is there a way of determining this path from inside the spider?
I don't know if scrapy is aware of where it is located on the filesystem, but in my settings.py
i add
PROJECT_ROOT = os.path.abspath(os.path.dirname(__file__))
then in spider you can say
from yourscrapyprojecthere import settings
settings.PROJECT_ROOT
Here I have assumed my project root is wherethe settings.py file is located. If you want to get the directory where scrapy.cfg
is you have to go up a directory
Very old question but for those often looking for the answer, scrapy has a built-in way of finding the project root:
from scrapy.utils.conf import closest_scrapy_cfg
proj_root = closest_scrapy_cfg()
will return the absolute path of the folder containing the scrapy.cfg file
EDIT: will return the absolute path of the scrapy.cfg file for the project
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With