I came across a situation where my scrapy code is working fine when used from command line but when I'm using the same spider after deploying (scrapy-deploy) and scheduling with scrapyd api it throws error in "scrapy.extensions.feedexport.FeedExporter" class.
1."open_spider" signal error
2016-05-14 12:09:38 [scrapy] INFO: Spider opened
2016-05-14 12:09:38 [scrapy] ERROR: Error caught on signal handler: <bound method ?.open_spider of <scrapy.extensions.feedexport.FeedExporter object at 0x7fafb1ce4a90>>
Traceback (most recent call last):
File "/home/jonsnow/venv/scrapy1/lib/python2.7/site-packages/twisted/internet/defer.py", line 150, in maybeDeferred
result = f(*args, **kw)
File "/home/jonsnow/venv/scrapy1/lib/python2.7/site-packages/pydispatch/robustapply.py", line 55, in robustApply
return receiver(*arguments, **named)
File "/home/jonsnow/venv/scrapy1/lib/python2.7/site-packages/scrapy/extensions/feedexport.py", line 185, in open_spider
uri = self.urifmt % self._get_uri_params(spider)
TypeError: float argument required, not dict
2."item_scraped" signal error:
2016-05-14 12:09:49 [scrapy] DEBUG: Scraped from <200 https://someurl.>
2016-05-14 12:09:49 [scrapy] ERROR: Error caught on signal handler: <bound method ?.item_scraped of <scrapy.extensions.feedexport.FeedExporter object at 0x7fafb1ce4a90>>
Traceback (most recent call last):
File "/home/jonsnow/venv/scrapy1/lib/python2.7/site-packages/twisted/internet/defer.py", line 150, in maybeDeferred
result = f(*args, **kw)
File "/home/jonsnow/venv/scrapy1/lib/python2.7/site-packages/pydispatch/robustapply.py", line 55, in robustApply
return receiver(*arguments, **named)
File "/home/jonsnow/venv/scrapy1/lib/python2.7/site-packages/scrapy/extensions/feedexport.py", line 210, in item_scraped
slot = self.slot
AttributeError: 'FeedExporter' object has no attribute 'slot'
3."close_spider" signal error:
2016-05-14 12:09:49 [scrapy] INFO: Closing spider (finished)
2016-05-14 12:09:49 [scrapy] ERROR: Error caught on signal handler: <bound method ?.close_spider of <scrapy.extensions.feedexport.FeedExporter object at 0x7fafb1ce4a90>>
Traceback (most recent call last):
File "/home/jonsnow/venv/scrapy1/lib/python2.7/site-packages/twisted/internet/defer.py", line 150, in maybeDeferred
result = f(*args, **kw)
File "/home/jonsnow/venv/scrapy1/lib/python2.7/site-packages/pydispatch/robustapply.py", line 55, in robustApply
return receiver(*arguments, **named)
File "/home/jonsnow/venv/scrapy1/lib/python2.7/site-packages/scrapy/extensions/feedexport.py", line 193, in close_spider
slot = self.slot
AttributeError: 'FeedExporter' object has no attribute 'slot'
also tried with scrapyd (1.1.0 with scrapy version 0.24.6)
It happens when feedexporter is not able to write file, it happend to me when i had my previously exported csv file open in excel. try to close the opened exported file it will work fine
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With