To get Scrapy working on HTTPS, I need PyOpenSSL, but I can't seem to get this to work.
So, is there a 64-bit version available? I don't see one... I have installed the 32-bit version but...
I currently get this error back from Scrapy:
ERROR: Error downloading <https://SOME_PRIVATE_URL_HERE>:
[Failure instance: Traceback: <class 'scrapy.exceptions.NotSupported'>:
HTTPS not supported: install pyopenssl library
Does anyone have a clue how to obtain or compile a 64-bit version? Or is that unneccesary?
Actually, step 9 at this website will resolve your issue.
http://steamforge.net/wiki/index.php/How_to_Install_Scrapy_in_64-bit_Windows_7
EDIT: Including the content from steamforge:
Notes
- Scrapy must be installed with Python 2.5, 2.6, or 2.7 (NOT 3.x)
- Python 2.7 (and 3.2) do not load the correct 32-bit compatible
Windows Registry Keys
Install Procedure
- Download & install Python 2.7.2 (using the Windows Installer) from
http://www.python.org/getit/
- Go to the Search programs and files bar at the bottom of the start
menu and type “regedit” and hit enter
-
Using the left pane navigate to
“HKEY_LOCAL_MACHINE\SOFTWARE\Python” & click on it (highlighting
it in blue)
- With the branch selected go to File --> Export
- Save the file somewhere you can get to it easily
- Open the file in your favorite text editor
- Insert "\Wow6432Node" inbetween “SOFTWARE” and "\Python" on
every line (an easy way to do this is to do a find and replace
with “SOFTWARE” in the Find Field and “SOFTWARE\Wow6432Node” in
the Replace Field
- Save the file
- Navigate to where you saved said file and double click on it to
initiate a Registry Modification
Now download the Scrapy 0.14 Windows installer from
http://pypi.python.org/pypi/Scrapy
- Double click the Scrapy Installer and everything should work fine.
- Download and install Twisted 11.1.0 for Python 2.7 64 bit from
http://twistedmatrix.com/trac/wiki/Downloads
-
Download and install Zope.Interface
- Download distribute_setup.py from
http://python-distribute.org/distribute_setup.py (right click
& save), then double click it to install
- Download zope.interface-3.8.0-py2.6-win-amd64.egg from
http://pypi.python.org/pypi/zope.interface/3.8.0#downloads
- Open a command prompt (type “cmd” into the same Search programs
and files pane as above) and navigate to your Python Scripts
folder (something like “cd C:\Python27\Scripts”)
- type “easy_install
C:\Downloads\zope.interface-3.8.0-py2.6-win-amd64.egg”
(replacing C:\Downloads\ with wherever you downloaded
Zope.Interface to) and hit enter
Download and install lxml-2.3.2.win-amd64-py2.7.exe from
http://www.lfd.uci.edu/~gohlke/pythonlibs/
-
Download and install
egenix-pyopenssl-0.13.0_1.0.0g_1.win-amd64-py2.7.msi from
http://www.egenix.com/cryptodownload/?file=egenix-pyopenssl-0.13.0_1.0.0g_1.win-amd64-py2.7.msi
- When it asks you to navigate to a Python 2.6 Directory, just
point it to your Python 2.7 Directory (C:\Python27 by default)
-
Install w3lib
- Open a command prompt (Start -> Search programs and files ->
“cmd”)
- Navigate to your Python Scripts Folder (nominally
“C:\Python27\Scripts”)
- Type “easy_install w3lib” and hit enter
-
Now we just need to add the Python Scripts folder to the System Path
so we can use scrapy
- Click the “Start Menu” button and Right Click on “Computer” and
select “Properties”
- Click “Advanced system settings” in the left bar
- Click “Environment Variables...”
- Find the “Path” variable in the (lower) System variables Pane
and double click on it
- Click in the “Variable value:” pane and hit the “End” button
- Type, or paste, ";C:\Python27\Scripts\" (modify for wherever
you installed Python) at the end of the line
- *Be sure not to disturb or change any of the existing information there
- Hit OK, OK,OK, X