SplashRequest gives - TypeError: attrs() got an unexpected keyword argument 'eq'

Question

I am using a cloud Splash instance from ScrapingHub. I am trying to do a simple request using the Scrapy-Splash library and I keep getting the error:

@attr.s(hash=False, repr=False, eq=False)
TypeError: attrs() got an unexpected keyword argument 'eq'

Any ideas/clues about why the error is showing up would be much appreciated.

The code I'm using looks like this (using Python 3.6 and Scrapy v 2.1.0):

import scrapy
from scrapy_splash import SplashRequest


class MySpider(scrapy.Spider):
    start_urls = ['https://www.sportsgirl.com.au', 'http://maryons.com.au']

    def start_requests(self):
        for url in self.start_urls:
            yield SplashRequest(url, self.parse, args={'wait': 0.5})

    def parse(self, response):    
        print(response)


MySpider().start_requests()

and settings.py has the following values:

DOWNLOADER_MIDDLEWARES = {
    'scrapy_splash.SplashCookiesMiddleware': 723,
    'scrapy_splash.SplashMiddleware': 725,
    'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware': 810,
}

SPIDER_MIDDLEWARES = {
    'scrapy_splash.SplashDeduplicateArgsMiddleware': 100,
}

SPLASH_URL = 'https://my-splash-url.scrapinghub.com'

Answer 1

The error is coming from the Twisted library, which is a dependency of Scrapy.

It looks like it's fixed in the latest version. If you run

pip install --upgrade twisted

you should be fine!

Answer 2

I also had to upgrade attrs in order to make the error message disappear.

pip3 install attrs==19.2.0 -i http://mirrors.aliyun.com/pypi/simple --trusted-host mirrors.aliyun.com

I found this on: https://www.programmersought.com/article/61587312947