Scrapy - No module named mail.smtp

Question

System: Ubuntu 14.04

I installed scrapy using the command sudo pip install scrapy.

I am following the tutorial located here.

When I run the command scrapy crawl dmoz at this step, I get the following error:

2015-07-04 15:28:58 [scrapy] INFO: Scrapy 1.0.1 started (bot: tutorial)
2015-07-04 15:28:58 [scrapy] INFO: Optional features available: ssl, http11
2015-07-04 15:28:58 [scrapy] INFO: Overridden settings: {'NEWSPIDER_MODULE': 'tutorial.spiders', 'SPIDER_MODULES': ['tutorial.spiders'], 'BOT_NAME': 'tutorial'}
Traceback (most recent call last):
  File "/usr/local/bin/scrapy", line 11, in <module>
    sys.exit(execute())
  File "/usr/local/lib/python2.7/dist-packages/scrapy/cmdline.py", line 143, in execute
    _run_print_help(parser, _run_command, cmd, args, opts)
  File "/usr/local/lib/python2.7/dist-packages/scrapy/cmdline.py", line 89, in _run_print_help
    func(*a, **kw)
  File "/usr/local/lib/python2.7/dist-packages/scrapy/cmdline.py", line 150, in _run_command
    cmd.run(args, opts)
  File "/usr/local/lib/python2.7/dist-packages/scrapy/commands/crawl.py", line 57, in run
    self.crawler_process.crawl(spname, **opts.spargs)
  File "/usr/local/lib/python2.7/dist-packages/scrapy/crawler.py", line 150, in crawl
    crawler = self._create_crawler(crawler_or_spidercls)
  File "/usr/local/lib/python2.7/dist-packages/scrapy/crawler.py", line 166, in _create_crawler
    return Crawler(spidercls, self.settings)
  File "/usr/local/lib/python2.7/dist-packages/scrapy/crawler.py", line 46, in __init__
    self.extensions = ExtensionManager.from_crawler(self)
  File "/usr/local/lib/python2.7/dist-packages/scrapy/middleware.py", line 56, in from_crawler
    return cls.from_settings(crawler.settings, crawler)
  File "/usr/local/lib/python2.7/dist-packages/scrapy/middleware.py", line 32, in from_settings
    mwcls = load_object(clspath)
  File "/usr/local/lib/python2.7/dist-packages/scrapy/utils/misc.py", line 44, in load_object
    mod = import_module(module)
  File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/usr/local/lib/python2.7/dist-packages/scrapy/extensions/memusage.py", line 16, in <module>
    from scrapy.mail import MailSender
  File "/usr/local/lib/python2.7/dist-packages/scrapy/mail.py", line 23, in <module>
    from twisted.mail.smtp import ESMTPSenderFactory
ImportError: No module named mail.smtp

Answer 1

I had same problem. installing twisted solved the problem.

sudo apt-get install python-twisted

Answer 2

Scrapy uses the twisted.mail.smtp module. Installing twisted should fix this issue

https://github.com/scrapy/scrapy/blob/342cb622f1ea93268477da557099010bbd72529a/docs/topics/email.rst