Follow to the redirect links in scrapy

Question

How i can set this? i have written handle_httpstatus_list = [301, 302, 303], but scrapy does not follow to the new links, it just has begun to save empty files of pages.

P.S. English is not my native language. Exuse me for that.

Answer 1

Scrapy does handle redirects by default, meaning that the RedirectMiddleware is activated in the DOWNLOADER_MIDDLEWARES setting and the REDIRECT_ENABLED setting is set to True.

Answer 2

You will also need to mention dont_redirect = true in your request as:

meta = {'dont_redirect': True, "handle_httpstatus_list" : [301, 302, 303]}

Regards