pip is not able to install packages correctly: Permission denied error [duplicate]

Question

I am trying to install lxml to install scrapy on my Mac (v 10.9.4)

╭─[email protected] ~ ╰─➤  pip install lxml Downloading/unpacking lxml   Downloading lxml-3.4.0.tar.gz (3.5MB): 3.5MB downloaded   Running setup.py (path:/private/var/folders/8l/t7tcq67d34v7qq_4hp3s1dm80000gn/T/pip_build_ishaantaylor/lxml/setup.py) egg_info for package lxml     /System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'bugtrack_url'       warnings.warn(msg)     Building lxml version 3.4.0.     Building without Cython.     Using build configuration of libxslt 1.1.28      warning: no previously-included files found matching '*.py' Installing collected packages: lxml   Running setup.py install for lxml     /System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'bugtrack_url'       warnings.warn(msg)     Building lxml version 3.4.0.     Building without Cython.     Using build configuration of libxslt 1.1.28     building 'lxml.etree' extension     cc -fno-strict-aliasing -fno-common -dynamic -arch x86_64 -arch i386 -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch x86_64 -arch i386 -pipe -I/usr/include/libxml2 -I/private/var/folders/8l/t7tcq67d34v7qq_4hp3s1dm80000gn/T/pip_build_ishaantaylor/lxml/src/lxml/includes -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c src/lxml/lxml.etree.c -o build/temp.macosx-10.9-intel-2.7/src/lxml/lxml.etree.o -w -flat_namespace     cc -bundle -undefined dynamic_lookup -arch x86_64 -arch i386 -Wl,-F. build/temp.macosx-10.9-intel-2.7/src/lxml/lxml.etree.o -lxslt -lexslt -lxml2 -lz -lm -o build/lib.macosx-10.9-intel-2.7/lxml/etree.so     building 'lxml.objectify' extension     cc -fno-strict-aliasing -fno-common -dynamic -arch x86_64 -arch i386 -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch x86_64 -arch i386 -pipe -I/usr/include/libxml2 -I/private/var/folders/8l/t7tcq67d34v7qq_4hp3s1dm80000gn/T/pip_build_ishaantaylor/lxml/src/lxml/includes -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c src/lxml/lxml.objectify.c -o build/temp.macosx-10.9-intel-2.7/src/lxml/lxml.objectify.o -w -flat_namespace     cc -bundle -undefined dynamic_lookup -arch x86_64 -arch i386 -Wl,-F. build/temp.macosx-10.9-intel-2.7/src/lxml/lxml.objectify.o -lxslt -lexslt -lxml2 -lz -lm -o build/lib.macosx-10.9-intel-2.7/lxml/objectify.so     error: could not create '/lib/python2.7/site-packages/lxml': Permission denied     Complete output from command /usr/bin/python -c "import setuptools, tokenize;__file__='/private/var/folders/8l/t7tcq67d34v7qq_4hp3s1dm80000gn/T/pip_build_ishaantaylor/lxml/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/8l/t7tcq67d34v7qq_4hp3s1dm80000gn/T/pip-_xm20j-record/install-record.txt --single-version-externally-managed --compile:     /System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'bugtrack_url'    warnings.warn(msg)  Building lxml version 3.4.0.  Building without Cython.  Using build configuration of libxslt 1.1.28  running install  running build  running build_py  creating build  creating build/lib.macosx-10.9-intel-2.7  creating build/lib.macosx-10.9-intel-2.7/lxml  copying src/lxml/__init__.py -> build/lib.macosx-10.9-intel-2.7/lxml  copying src/lxml/_elementpath.py -> build/lib.macosx-10.9-intel-2.7/lxml  copying src/lxml/builder.py -> build/lib.macosx-10.9-intel-2.7/lxml  copying src/lxml/cssselect.py -> build/lib.macosx-10.9-intel-2.7/lxml  copying src/lxml/doctestcompare.py -> build/lib.macosx-10.9-intel-2.7/lxml  copying src/lxml/ElementInclude.py -> build/lib.macosx-10.9-intel-2.7/lxml  copying src/lxml/pyclasslookup.py -> build/lib.macosx-10.9-intel-2.7/lxml  copying src/lxml/sax.py -> build/lib.macosx-10.9-intel-2.7/lxml  copying src/lxml/usedoctest.py -> build/lib.macosx-10.9-intel-2.7/lxml  creating build/lib.macosx-10.9-intel-2.7/lxml/includes  copying src/lxml/includes/__init__.py -> build/lib.macosx-10.9-intel-2.7/lxml/includes  creating build/lib.macosx-10.9-intel-2.7/lxml/html  copying src/lxml/html/__init__.py -> build/lib.macosx-10.9-intel-2.7/lxml/html  copying src/lxml/html/_diffcommand.py -> build/lib.macosx-10.9-intel-2.7/lxml/html  copying src/lxml/html/_html5builder.py -> build/lib.macosx-10.9-intel-2.7/lxml/html  copying src/lxml/html/_setmixin.py -> build/lib.macosx-10.9-intel-2.7/lxml/html  copying src/lxml/html/builder.py -> build/lib.macosx-10.9-intel-2.7/lxml/html  copying src/lxml/html/clean.py -> build/lib.macosx-10.9-intel-2.7/lxml/html  copying src/lxml/html/defs.py -> build/lib.macosx-10.9-intel-2.7/lxml/html  copying src/lxml/html/diff.py -> build/lib.macosx-10.9-intel-2.7/lxml/html  copying src/lxml/html/ElementSoup.py -> build/lib.macosx-10.9-intel-2.7/lxml/html  copying src/lxml/html/formfill.py -> build/lib.macosx-10.9-intel-2.7/lxml/html  copying src/lxml/html/html5parser.py -> build/lib.macosx-10.9-intel-2.7/lxml/html  copying src/lxml/html/soupparser.py -> build/lib.macosx-10.9-intel-2.7/lxml/html  copying src/lxml/html/usedoctest.py -> build/lib.macosx-10.9-intel-2.7/lxml/html  creating build/lib.macosx-10.9-intel-2.7/lxml/isoschematron  copying src/lxml/isoschematron/__init__.py -> build/lib.macosx-10.9-intel-2.7/lxml/isoschematron  copying src/lxml/lxml.etree.h -> build/lib.macosx-10.9-intel-2.7/lxml  copying src/lxml/lxml.etree_api.h -> build/lib.macosx-10.9-intel-2.7/lxml  copying src/lxml/includes/c14n.pxd -> build/lib.macosx-10.9-intel-2.7/lxml/includes  copying src/lxml/includes/config.pxd -> build/lib.macosx-10.9-intel-2.7/lxml/includes  copying src/lxml/includes/dtdvalid.pxd -> build/lib.macosx-10.9-intel-2.7/lxml/includes  copying src/lxml/includes/etreepublic.pxd -> build/lib.macosx-10.9-intel-2.7/lxml/includes  copying src/lxml/includes/htmlparser.pxd -> build/lib.macosx-10.9-intel-2.7/lxml/includes  copying src/lxml/includes/relaxng.pxd -> build/lib.macosx-10.9-intel-2.7/lxml/includes  copying src/lxml/includes/schematron.pxd -> build/lib.macosx-10.9-intel-2.7/lxml/includes  copying src/lxml/includes/tree.pxd -> build/lib.macosx-10.9-intel-2.7/lxml/includes  copying src/lxml/includes/uri.pxd -> build/lib.macosx-10.9-intel-2.7/lxml/includes  copying src/lxml/includes/xinclude.pxd -> build/lib.macosx-10.9-intel-2.7/lxml/includes  copying src/lxml/includes/xmlerror.pxd -> build/lib.macosx-10.9-intel-2.7/lxml/includes  copying src/lxml/includes/xmlparser.pxd -> build/lib.macosx-10.9-intel-2.7/lxml/includes  copying src/lxml/includes/xmlschema.pxd -> build/lib.macosx-10.9-intel-2.7/lxml/includes  copying src/lxml/includes/xpath.pxd -> build/lib.macosx-10.9-intel-2.7/lxml/includes  copying src/lxml/includes/xslt.pxd -> build/lib.macosx-10.9-intel-2.7/lxml/includes  copying src/lxml/includes/etree_defs.h -> build/lib.macosx-10.9-intel-2.7/lxml/includes  copying src/lxml/includes/lxml-version.h -> build/lib.macosx-10.9-intel-2.7/lxml/includes  creating build/lib.macosx-10.9-intel-2.7/lxml/isoschematron/resources  creating build/lib.macosx-10.9-intel-2.7/lxml/isoschematron/resources/rng  copying src/lxml/isoschematron/resources/rng/iso-schematron.rng -> build/lib.macosx-10.9-intel-2.7/lxml/isoschematron/resources/rng  creating build/lib.macosx-10.9-intel-2.7/lxml/isoschematron/resources/xsl  copying src/lxml/isoschematron/resources/xsl/RNG2Schtrn.xsl -> build/lib.macosx-10.9-intel-2.7/lxml/isoschematron/resources/xsl  copying src/lxml/isoschematron/resources/xsl/XSD2Schtrn.xsl -> build/lib.macosx-10.9-intel-2.7/lxml/isoschematron/resources/xsl  creating build/lib.macosx-10.9-intel-2.7/lxml/isoschematron/resources/xsl/iso-schematron-xslt1  copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_abstract_expand.xsl -> build/lib.macosx-10.9-intel-2.7/lxml/isoschematron/resources/xsl/iso-schematron-xslt1  copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_dsdl_include.xsl -> build/lib.macosx-10.9-intel-2.7/lxml/isoschematron/resources/xsl/iso-schematron-xslt1  copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_schematron_message.xsl -> build/lib.macosx-10.9-intel-2.7/lxml/isoschematron/resources/xsl/iso-schematron-xslt1  copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_schematron_skeleton_for_xslt1.xsl -> build/lib.macosx-10.9-intel-2.7/lxml/isoschematron/resources/xsl/iso-schematron-xslt1  copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_svrl_for_xslt1.xsl -> build/lib.macosx-10.9-intel-2.7/lxml/isoschematron/resources/xsl/iso-schematron-xslt1  copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/readme.txt -> build/lib.macosx-10.9-intel-2.7/lxml/isoschematron/resources/xsl/iso-schematron-xslt1  running build_ext  building 'lxml.etree' extension  creating build/temp.macosx-10.9-intel-2.7  creating build/temp.macosx-10.9-intel-2.7/src  creating build/temp.macosx-10.9-intel-2.7/src/lxml  cc -fno-strict-aliasing -fno-common -dynamic -arch x86_64 -arch i386 -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch x86_64 -arch i386 -pipe -I/usr/include/libxml2 -I/private/var/folders/8l/t7tcq67d34v7qq_4hp3s1dm80000gn/T/pip_build_ishaantaylor/lxml/src/lxml/includes -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c src/lxml/lxml.etree.c -o build/temp.macosx-10.9-intel-2.7/src/lxml/lxml.etree.o -w -flat_namespace  cc -bundle -undefined dynamic_lookup -arch x86_64 -arch i386 -Wl,-F. build/temp.macosx-10.9-intel-2.7/src/lxml/lxml.etree.o -lxslt -lexslt -lxml2 -lz -lm -o build/lib.macosx-10.9-intel-2.7/lxml/etree.so  building 'lxml.objectify' extension  cc -fno-strict-aliasing -fno-common -dynamic -arch x86_64 -arch i386 -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch x86_64 -arch i386 -pipe -I/usr/include/libxml2 -I/private/var/folders/8l/t7tcq67d34v7qq_4hp3s1dm80000gn/T/pip_build_ishaantaylor/lxml/src/lxml/includes -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c src/lxml/lxml.objectify.c -o build/temp.macosx-10.9-intel-2.7/src/lxml/lxml.objectify.o -w -flat_namespace  cc -bundle -undefined dynamic_lookup -arch x86_64 -arch i386 -Wl,-F. build/temp.macosx-10.9-intel-2.7/src/lxml/lxml.objectify.o -lxslt -lexslt -lxml2 -lz -lm -o build/lib.macosx-10.9-intel-2.7/lxml/objectify.so  running install_lib  creating /lib/python2.7/site-packages/lxml  error: could not create '/lib/python2.7/site-packages/lxml': Permission denied  ---------------------------------------- Cleaning up... Command /usr/bin/python -c "import setuptools, tokenize;__file__='/private/var/folders/8l/t7tcq67d34v7qq_4hp3s1dm80000gn/T/pip_build_ishaantaylor/lxml/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/8l/t7tcq67d34v7qq_4hp3s1dm80000gn/T/pip-_xm20j-record/install-record.txt --single-version-externally-managed --compile failed with error code 1 in /private/var/folders/8l/t7tcq67d34v7qq_4hp3s1dm80000gn/T/pip_build_ishaantaylor/lxml Storing debug log for failure in /Users/ishaantaylor/Library/Logs/pip.log 

If somebody could help me I would really appreciate it! I have a suspicion that the issue has to do with multiple versions of python but I'm not sure how to check or fix that.

Edit 1: When I try sudo pip install lxml:

╭─[email protected] ~ ╰─➤  sudo pip install lxml Traceback (most recent call last):   File "/bin/pip", line 7, in <module>     from pip import main ImportError: No module named pip 

Edit 2: When I try STATIC_DEPS=true sudo pip install lxml

╭─[email protected] ~ ╰─➤  STATIC_DEPS=true sudo pip install lxml Traceback (most recent call last):   File "/bin/pip", line 7, in <module>     from pip import main ImportError: No module named pip 

Also, I tried installing another package with pip and that didn't work either.. see below:

╭─[email protected] /lib/python2.7/site-packages ╰─➤  pip install requests Downloading/unpacking requests   Downloading requests-2.4.1-py2.py3-none-any.whl (458kB): 458kB downloaded Installing collected packages: requests Cleaning up... Exception: Traceback (most recent call last):   File "/usr/local/lib/python2.7/site-packages/pip-1.5.6-py2.7.egg/pip/basecommand.py", line 122, in main     status = self.run(options, args)   File "/usr/local/lib/python2.7/site-packages/pip-1.5.6-py2.7.egg/pip/commands/install.py", line 283, in run     requirement_set.install(install_options, global_options, root=options.root_path)   File "/usr/local/lib/python2.7/site-packages/pip-1.5.6-py2.7.egg/pip/req.py", line 1435, in install     requirement.install(install_options, global_options, *args, **kwargs)   File "/usr/local/lib/python2.7/site-packages/pip-1.5.6-py2.7.egg/pip/req.py", line 671, in install     self.move_wheel_files(self.source_dir, root=root)   File "/usr/local/lib/python2.7/site-packages/pip-1.5.6-py2.7.egg/pip/req.py", line 901, in move_wheel_files     pycompile=self.pycompile,   File "/usr/local/lib/python2.7/site-packages/pip-1.5.6-py2.7.egg/pip/wheel.py", line 215, in move_wheel_files     clobber(source, lib_dir, True)   File "/usr/local/lib/python2.7/site-packages/pip-1.5.6-py2.7.egg/pip/wheel.py", line 205, in clobber     os.makedirs(destdir)   File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/os.py", line 157, in makedirs     mkdir(name, mode) OSError: [Errno 13] Permission denied: '/lib/python2.7/site-packages/requests' Storing debug log for failure in /Users/ishaantaylor/Library/Logs/pip.log 

Edit 4: Tried to download virtualenv, people recommended this as an alternative to sudo. Result is below:

╭─[email protected] ~ ╰─➤  pip install virtualenv                                                               127 ↵ Downloading/unpacking virtualenv   Downloading virtualenv-1.11.6-py2.py3-none-any.whl (1.6MB): 1.6MB downloaded Installing collected packages: virtualenv Cleaning up... Exception: Traceback (most recent call last):   File "/usr/local/lib/python2.7/site-packages/pip-1.5.6-py2.7.egg/pip/basecommand.py", line 122, in main     status = self.run(options, args)   File "/usr/local/lib/python2.7/site-packages/pip-1.5.6-py2.7.egg/pip/commands/install.py", line 283, in run     requirement_set.install(install_options, global_options, root=options.root_path)   File "/usr/local/lib/python2.7/site-packages/pip-1.5.6-py2.7.egg/pip/req.py", line 1435, in install     requirement.install(install_options, global_options, *args, **kwargs)   File "/usr/local/lib/python2.7/site-packages/pip-1.5.6-py2.7.egg/pip/req.py", line 671, in install     self.move_wheel_files(self.source_dir, root=root)   File "/usr/local/lib/python2.7/site-packages/pip-1.5.6-py2.7.egg/pip/req.py", line 901, in move_wheel_files     pycompile=self.pycompile,   File "/usr/local/lib/python2.7/site-packages/pip-1.5.6-py2.7.egg/pip/wheel.py", line 215, in move_wheel_files     clobber(source, lib_dir, True)   File "/usr/local/lib/python2.7/site-packages/pip-1.5.6-py2.7.egg/pip/wheel.py", line 209, in clobber     shutil.copy2(srcfile, destfile)   File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 130, in copy2     copyfile(src, dst)   File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 83, in copyfile     with open(dst, 'wb') as fdst: IOError: [Errno 13] Permission denied: '/lib/python2.7/site-packages/virtualenv.py'  Storing debug log for failure in /Users/ishaantaylor/Library/Logs/pip.log 

Edit 5:

╭─[email protected] ~ ╰─➤  sudo pip install virtualenv                                                            1 ↵ Password: Traceback (most recent call last):   File "/bin/pip", line 7, in <module>     from pip import main ImportError: No module named pip ╭─[email protected] ~ ╰─➤  pip install virtualenv                                                                 1 ↵ Downloading/unpacking virtualenv   Downloading virtualenv-1.11.6-py2.py3-none-any.whl (1.6MB): 1.6MB downloaded Installing collected packages: virtualenv Cleaning up... Exception: Traceback (most recent call last):   File "/usr/local/lib/python2.7/site-packages/pip-1.5.6-py2.7.egg/pip/basecommand.py", line 122, in main     status = self.run(options, args)   File "/usr/local/lib/python2.7/site-packages/pip-1.5.6-py2.7.egg/pip/commands/install.py", line 283, in run     requirement_set.install(install_options, global_options, root=options.root_path)   File "/usr/local/lib/python2.7/site-packages/pip-1.5.6-py2.7.egg/pip/req.py", line 1435, in install     requirement.install(install_options, global_options, *args, **kwargs)   File "/usr/local/lib/python2.7/site-packages/pip-1.5.6-py2.7.egg/pip/req.py", line 671, in install     self.move_wheel_files(self.source_dir, root=root)   File "/usr/local/lib/python2.7/site-packages/pip-1.5.6-py2.7.egg/pip/req.py", line 901, in move_wheel_files     pycompile=self.pycompile,   File "/usr/local/lib/python2.7/site-packages/pip-1.5.6-py2.7.egg/pip/wheel.py", line 215, in move_wheel_files     clobber(source, lib_dir, True)   File "/usr/local/lib/python2.7/site-packages/pip-1.5.6-py2.7.egg/pip/wheel.py", line 209, in clobber     shutil.copy2(srcfile, destfile)   File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 130, in copy2     copyfile(src, dst)   File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 83, in copyfile     with open(dst, 'wb') as fdst: IOError: [Errno 13] Permission denied: '/lib/python2.7/site-packages/virtualenv.py'  Storing debug log for failure in /Users/ishaantaylor/Library/Logs/pip.log 

Edit 6: In response to hd1:

╭─[email protected] ~ ╰─➤  curl -kLso /tmp/get-pip.py https://bootstrap.pypa.io/get-pip.py ╭─[email protected] ~ ╰─➤  sudo python get-pip.py python: can't open file 'get-pip.py': [Errno 2] No such file or directory 

Edit 7: In Response to hd1:

╭─[email protected] / ╰─➤  curl -kLso /tmp/get-pip.py https://bootstrap.pypa.io/get-pip.py ╭─[email protected] / ╰─➤  cd ~ ╭─[email protected] ~ ╰─➤  sudo python /tmp/get-pip.py Requirement already up-to-date: pip in /usr/local/lib/python2.7/site-packages/pip-1.5.6-py2.7.egg Cleaning up... ╭─[email protected] ~ ╰─➤  sudo pip install virtualenv Downloading/unpacking virtualenv   Downloading virtualenv-1.11.6-py2.py3-none-any.whl (1.6MB): 1.6MB downloaded Installing collected packages: virtualenv Successfully installed virtualenv Cleaning up... ╭─[email protected] ~ ╰─➤  mkdir -p ~/.virtualenvs ╭─[email protected] ~ ╰─➤  virtualenv ~/.virtualenvs/lxmltest Traceback (most recent call last):   File "/bin/virtualenv", line 7, in <module>     from virtualenv import main ImportError: No module named virtualenv ╭─[email protected] ~ ╰─➤  which virtualenv                                                       1 ↵ /bin/virtualenv ╭─[email protected] ~ ╰─➤ echo -e ${PATH//:/\\n} /usr/local/bin /usr/bin /bin /usr/sbin /sbin /opt/X11/bin 

Answer 1

It looks like you're having a permissions error, based on this message in your output: error: could not create '/lib/python2.7/site-packages/lxml': Permission denied.

One thing you can try is doing a user install of the package with pip install lxml --user. For more information on how that works, check out this StackOverflow answer. (Thanks to Ishaan Taylor for the suggestion)

You can also run pip install as a superuser with sudo pip install lxml but it is not generally a good idea because it can cause issues with your system-level packages.