I know that with the command:
scrapy crawl somespider -s JOBDIR=crawls/somespider-1
I can pause/resume scrapes with CTRL+C.
What I want to know is how I can reset the scrapy and start back at the beginning. Is there a file I need to delete/empty?
M.
Yes, you should remove your JOBDIR
scrapy crawl somespider -s JOBDIR=crawls/somespider-1
<ctr+C>
rm -rf crawls/somespider-1
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With