Error Code 1 Installing Scrapy

Question

I am trying to install the Scrapy module for Python. I am using the command:

pip install Scrapy

But I get the following error:

Command "/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python -c "import setuptools, tokenize;__file__='/private/var/folders/b4/nrmj5jf105lcv1_kf1yk9gh40000gn/T/pip-build-aNQoXA/lxml/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/b4/nrmj5jf105lcv1_kf1yk9gh40000gn/T/pip-B_akAd-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /private/var/folders/b4/nrmj5jf105lcv1_kf1yk9gh40000gn/T/pip-build-aNQoXA/lxml

I've done some research on this, and tried several different solutions here and here but to no avail.

Answer 1

you can install it via pip wheel.
Download from here wheel package. Scrapy

Then install with

pip install wheel       #  to install support for wheel files.
pip install Scrapy‑1.0.3‑py2‑none‑any.whl        # to install the wheel.

EDIT: If on ubuntu or another debian based os:

try:

sudo apt-get install python-dev

sudo apt-get install libxml2-dev libxslt1-dev

and then again try to install scrapy.

On Mac Osx :

Install brew package manager from :Brew

Then try to use brew install package_name

Answer 2

Apparently I needed to install the Xcode command line tools. (Even if you have Xcode installed.) I have the latest beta of Xcode, but I still needed to install the command line tools.

xcode-select --install

And then running pip install scrapy worked fine.