Funnelweb error, cannot import OrderedDict

Question

I am trying to import some static pages from a site using funnelweb add-on for plone. Here is what I write: bin/funnelweb --crawler:url=http://wiki.scandiatransplant.com --crawler:max=50 --ploneupload=http://admin:admin@localhost:8080/TestPage.

I get get this error:

ImportError: cannot import name OrderedDict

The full traceback is added below. Does anyone know how to solve this?

Traceback (most recent call last):
  File "bin/funnelweb", line 116, in <module>
    mr.migrator.runner.runner({},"funnelweb.remote")
  File "/home/magiq/Plone/buildout-cache/eggs/mr.migrator-1.0.1-py2.6.egg/mr/migrator/runner/__init__.py", line 69, in runner
    load_config('autoinclude.zcml', mr.migrator)
  File "/home/magiq/Plone/buildout-cache/eggs/Zope2-2.13.12-py2.6.egg/Zope2/App/zcml.py", line 55, in load_config
    _context = xmlconfig.file(config, package, _context, execute=execute)
  File "/home/magiq/Plone/buildout-cache/eggs/zope.configuration-3.7.4-py2.6.egg/zope/configuration/xmlconfig.py", line 649, in file
    include(context, name, package)
  File "/home/magiq/Plone/buildout-cache/eggs/zope.configuration-3.7.4-py2.6.egg/zope/configuration/xmlconfig.py", line 548, in include
    processxmlfile(f, context)
  File "/home/magiq/Plone/buildout-cache/eggs/zope.configuration-3.7.4-py2.6.egg/zope/configuration/xmlconfig.py", line 380, in processxmlfile
    parser.parse(src)
  File "/home/magiq/Plone/Python-2.6/lib/python2.6/xml/sax/expatreader.py", line 107, in parse
    xmlreader.IncrementalParser.parse(self, source)
  File "/home/magiq/Plone/Python-2.6/lib/python2.6/xml/sax/xmlreader.py", line 123, in parse
    self.feed(buffer)
  File "/home/magiq/Plone/Python-2.6/lib/python2.6/xml/sax/expatreader.py", line 207, in feed
    self._parser.Parse(data, isFinal)
  File "/home/magiq/Plone/Python-2.6/lib/python2.6/xml/sax/expatreader.py", line 349, in end_element_ns
    self._cont_handler.endElementNS(pair, None)
  File "/home/magiq/Plone/buildout-cache/eggs/zope.configuration-3.7.4-py2.6.egg/zope/configuration/xmlconfig.py", line 359, in endElementNS
    self.context.end()
  File "/home/magiq/Plone/buildout-cache/eggs/zope.configuration-3.7.4-py2.6.egg/zope/configuration/config.py", line 558, in end
    self.stack.pop().finish()
  File "/home/magiq/Plone/buildout-cache/eggs/zope.configuration-3.7.4-py2.6.egg/zope/configuration/config.py", line 706, in finish
    actions = self.handler(context, **args)
  File "/home/magiq/Plone/buildout-cache/eggs/z3c.autoinclude-0.3.4-py2.6.egg/z3c/autoinclude/zcml.py", line 104, in includePluginsDirective
    includeZCMLGroup(_context, info, filename)
  File "/home/magiq/Plone/buildout-cache/eggs/z3c.autoinclude-0.3.4-py2.6.egg/z3c/autoinclude/zcml.py", line 30, in includeZCMLGroup
    include(_context, filename, includable_package)
  File "/home/magiq/Plone/buildout-cache/eggs/zope.configuration-3.7.4-py2.6.egg/zope/configuration/xmlconfig.py", line 548, in include
    processxmlfile(f, context)
  File "/home/magiq/Plone/buildout-cache/eggs/zope.configuration-3.7.4-py2.6.egg/zope/configuration/xmlconfig.py", line 380, in processxmlfile
    parser.parse(src)
  File "/home/magiq/Plone/Python-2.6/lib/python2.6/xml/sax/expatreader.py", line 107, in parse
    xmlreader.IncrementalParser.parse(self, source)
  File "/home/magiq/Plone/Python-2.6/lib/python2.6/xml/sax/xmlreader.py", line 123, in parse
    self.feed(buffer)
  File "/home/magiq/Plone/Python-2.6/lib/python2.6/xml/sax/expatreader.py", line 207, in feed
    self._parser.Parse(data, isFinal)
  File "/home/magiq/Plone/Python-2.6/lib/python2.6/xml/sax/expatreader.py", line 349, in end_element_ns
    self._cont_handler.endElementNS(pair, None)
  File "/home/magiq/Plone/buildout-cache/eggs/zope.configuration-3.7.4-py2.6.egg/zope/configuration/xmlconfig.py", line 359, in endElementNS
    self.context.end()
  File "/home/magiq/Plone/buildout-cache/eggs/zope.configuration-3.7.4-py2.6.egg/zope/configuration/config.py", line 558, in end
    self.stack.pop().finish()
  File "/home/magiq/Plone/buildout-cache/eggs/zope.configuration-3.7.4-py2.6.egg/zope/configuration/config.py", line 705, in finish
    args = toargs(context, *self.argdata)
  File "/home/magiq/Plone/buildout-cache/eggs/zope.configuration-3.7.4-py2.6.egg/zope/configuration/config.py", line 1397, in toargs
    args[str(name)] = field.fromUnicode(s)
  File "/home/magiq/Plone/buildout-cache/eggs/zope.configuration-3.7.4-py2.6.egg/zope/configuration/fields.py", line 137, in fromUnicode
    value = self.context.resolve(name)
  File "/home/magiq/Plone/buildout-cache/eggs/zope.configuration-3.7.4-py2.6.egg/zope/configuration/config.py", line 179, in resolve
    mod = __import__(mname, *_import_chickens)
  File "/home/magiq/Plone/buildout-cache/eggs/transmogrify.webcrawler-1.2.1-py2.6.egg/transmogrify/webcrawler/webcrawler.py", line 21, in <module>
    from collections import OrderedDict
zope.configuration.xmlconfig.ZopeXMLConfigurationError: File "/home/magiq/Plone/buildout-cache/eggs/mr.migrator-1.0.1-py2.6.egg/mr/migrator/autoinclude.zcml", line 10.4-10.85
    ZopeXMLConfigurationError: File "/home/magiq/Plone/buildout-cache/eggs/transmogrify.webcrawler-1.2.1-py2.6.egg/transmogrify/webcrawler/configure.zcml", line 9.4-10.46
    ImportError: cannot import name OrderedDict

Answer 1

OrderedDict is a new class added to Python 2.7, so it is missing in your case.

You can install a backport:

pip install ordereddict

or add that as a dependency in your buildout, and edit Plone/buildout-cache/eggs/transmogrify.webcrawler-1.2.1-py2.6.egg/transmogrify/webcrawler/webcrawler.py line 21 to change the import from:

from collections import OrderedDict

to

try:
    from collections import OrderedDict
except ImportError:
    # python 2.6 or earlier, use backport
    from ordereddict import OrderedDict

I have filed an issue in the funnelweb issue tracker to request that this is added to a future release of the package, see issue 22.