I use try except
to avoid error,but My terminal still show error but not the log message :
raise ValueError('Missing scheme in request url: %s' % self._url)
exceptions.ValueError: Missing scheme in request url:
How can I avoid this error when scrapy didn't get image_urls?
Please guide me ,thank you very much.
try:
item['image_urls'] = ["".join(image.extract()) ]
except:
log.msg("no image foung!. url={}".format(response.url),level=log.INFO)
the image_urls
field should be a list, not a str.
item['image_urls'] = image.extract()
If you do so, and still raise the exception, it seems that the urls you scraped is the relative path.
the ImagePipeline
doesn't know your host, so it can not generate the absolute path to crawl.
import urlparse
item['image_urls'] = [ urlparse.urljoin(response.url, u) for u in image.extract() ]
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With