install scrapy on win 7 (64-bit)

Question

I'm trying to install scrapy for python2.6, but it seems not going well. Here is the packages installed:

G:\Python26\Scripts>pip freeze
Scrapy==0.16.4
Twisted==12.3.0
libxml2-python==2.7.7
lxml==2.3.6
pyopenssl==0.13
w3lib==1.2
zope.interface==3.8.0

I also got iconv and zlib. And this is the log when installing scrapy with pip. I don't know what I should do next, am I missing sth? Need instructions, thank you.

win 7 64-bit, Visual C++ installed

C:\Users\d>pip install scrapy
Downloading/unpacking scrapy
  Downloading Scrapy-0.16.4.tar.gz (679kB): 679kB downloaded
  Running setup.py egg_info for package scrapy
    warning: no files found matching 'license.txt' under directory 'scrapy'
    no previously-included directories found matching 'docs\build'
Requirement already satisfied (use --upgrade to upgrade): Twisted>=8.0 in g:\pyt
hon26\lib\site-packages (from scrapy)
Requirement already satisfied (use --upgrade to upgrade): w3lib>=1.2 in g:\pytho
n26\lib\site-packages (from scrapy)
Requirement already satisfied (use --upgrade to upgrade): lxml in g:\python26\li
b\site-packages\lxml-2.3.6-py2.6-win32.egg (from scrapy)
Requirement already satisfied (use --upgrade to upgrade): pyOpenSSL in g:\python
26\lib\site-packages\pyopenssl-0.13-py2.6-win32.egg (from scrapy)
Requirement already satisfied (use --upgrade to upgrade): zope.interface>=3.6.0
in g:\python26\lib\site-packages\zope.interface-3.8.0-py2.6-win32.egg (from Twis
ted>=8.0->scrapy)
Requirement already satisfied (use --upgrade to upgrade): setuptools in g:\pytho
n26\lib\site-packages (from zope.interface>=3.6.0->Twisted>=8.0->scrapy)
Installing collected packages: scrapy
  Running setup.py install for scrapy
    warning: no files found matching 'license.txt' under directory 'scrapy'
    no previously-included directories found matching 'docs\build'
Successfully installed scrapy
Cleaning up...


C:\Users\d>scrapy
Traceback (most recent call last):
  File "G:\Python26\lib\runpy.py", line 122, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "G:\Python26\lib\runpy.py", line 34, in _run_code
    exec code in run_globals
  File "G:\Python26\lib\site-packages\scrapy\cmdline.py", line 7, in <module>
    from scrapy.crawler import CrawlerProcess
  File "G:\Python26\lib\site-packages\scrapy\crawler.py", line 5, in <module>
    from scrapy.core.engine import ExecutionEngine
  File "G:\Python26\lib\site-packages\scrapy\core\engine.py", line 14, in <modul
e>
    from scrapy.core.downloader import Downloader
  File "G:\Python26\lib\site-packages\scrapy\core\downloader\__init__.py", line
16, in <module>
    from .middleware import DownloaderMiddlewareManager
  File "G:\Python26\lib\site-packages\scrapy\core\downloader\middleware.py", lin
e 7, in <module>
    from scrapy.http import Request, Response
  File "G:\Python26\lib\site-packages\scrapy\http\__init__.py", line 11, in <mod
ule>
    from scrapy.http.request.form import FormRequest
  File "G:\Python26\lib\site-packages\scrapy\http\request\form.py", line 9, in <
module>
    import lxml.html
  File "G:\Python26\lib\site-packages\lxml-2.3.6-py2.6-win32.egg\lxml\html\__ini
t__.py", line 12, in <module>
    from lxml import etree
ImportError: DLL load failed: %1 is not a valid Win32 application

Answer 1

You need to follow the instructions here (also listed above).

The specific DLL error listed in the question is due (at least in my case) to installing the 32 bit OpenSSL with 64 bit everything else. After I installed the correct OpenSSL (and, incidentally, had to uninstall all of the packages in the link above and start over- order matters for the dependency msi installers), scrapy is working fine on python27, windows7, 64 bit.