Get "ParseError" with Scrapy

Question

Most time scraper works normally, but sometimes I get

2014-08-28 11:02:27-0700 [aqicn] DEBUG: Retrying <GET http://site_address_com/> (failed 1 times): [<twisted.python.failure.Failure <class 'twisted.web._newclient.ParseError'>>]

I cannot understand why it happening. Who have same problem? May be it twisted bug? I try find something, but I got only partial official documentation.

Answer 1

I think you are running into this bug. As you can see there is no official fix right now, just a monkey patch on the ticket.

Answer 2

As were thinking @nramirezuy and @B.Mr.W scraper was banned on site. I use TOR and now I don't get this error. Thank you guys for help.