scrapy yield Request not working

Question

I wrote the following scrapy spider but it's not continuing the crawling process after the initial request, although I've yielded more scrapy.Requests for scrapy to follow.

import regex as re
import scrapy
from scrapy.linkextractors import LinkExtractor
from scrapy.spiders import Spider

class myspider(Spider):
name = 'haha'

allowed_domains = ['https://blog.scrapinghub.com/']
start_urls = ['https://blog.scrapinghub.com/']
extractor = LinkExtractor(allow=allowed_domains)

def parse(self, response):
    # To extract all the links on this page
    links_in_page = self.extractor.extract_links(response)

    for link in links_in_page:
        yield scrapy.Request(link.url, callback=self.parse)

Answer 1

allowed_domains needs to be a list of domains, not a list of URLs.

So it should be:

allowed_domains = ['blog.scrapinghub.com']