How restart Scrapy spider

Question

What I need:

1. start crawler
2. crawler job done
3. wait 1 minute
4. start crawler again

I try this:

from scrapy.crawler import CrawlerProcess
from scrapy.utils.project import get_project_settings
from time import sleep

while True:
    process = CrawlerProcess(get_project_settings())
    process.crawl('spider_name')
    process.start()
    sleep(60)

But get error:

twisted.internet.error.ReactorNotRestartable

please help me do it right

Python 3.6
Scrapy 1.3.2
Linux

Answer 1

I think I found the solution:

from scrapy.utils.project import get_project_settings
from scrapy.crawler import CrawlerRunner
from twisted.internet import reactor
from twisted.internet import task


timeout = 60


def run_spider():
    l.stop()
    runner = CrawlerRunner(get_project_settings())
    d = runner.crawl('spider_name')
    d.addBoth(lambda _: l.start(timeout, False))


l = task.LoopingCall(run_spider)
l.start(timeout)

reactor.run()