Using multiple spiders at in the project in Scrapy

Question

I wanna know if it is possible to use multiple spiders within the same project together. Actually I need 2 spiders. The first one gathers the links on which the second spider should scrape. They both work on the same website, so the domain is similar.Is it possible? If yes can you give me an example? Thanks

Answer 1

Maybe this is what you're looking for:

def parse(self, response):
    # parse the links (aka your first spider)
    for link in hxs('//XPATH'):
        yield Request(link.extract(), callback=self.parse_link)

def parse_link(self, response):
    # continue parsing (aka your second spider)

Hope this help you :)