Windows + virtualenv + pip + NumPy (problems when installing NumPy)

Question

On Windows, I normally just use the binary installer, but I would like to install NumPy only in a virtualenv this time, so I created a virtual env:

virtualenv --no-site-packages --distribute summary_python cd summary_python/Scripts activate.bat 

Then I tried to install NumPy

pip install numpy 

And I get an error. My pip.log is pasted below:

Downloading/unpacking numpy   Running setup.py egg_info for package numpy     non-existing path in 'numpy\\distutils': 'site.cfg'     F2PY Version 2     blas_opt_info:     blas_mkl_info:       libraries mkl,vml,guide not found in c:\Users\fname.lname\Documents\summary_python\lib       libraries mkl,vml,guide not found in C:\       NOT AVAILABLE      atlas_blas_threads_info:     Setting PTATLAS=ATLAS       libraries ptf77blas,ptcblas,atlas not found in c:\Users\fname.lname\Documents\summary_python\lib       libraries ptf77blas,ptcblas,atlas not found in C:\       NOT AVAILABLE      atlas_blas_info:       libraries f77blas,cblas,atlas not found in c:\Users\fname.lname\Documents\summary_python\lib       libraries f77blas,cblas,atlas not found in C:\       NOT AVAILABLE      blas_info:       libraries blas not found in c:\Users\fname.lname\Documents\summary_python\lib       libraries blas not found in C:\       NOT AVAILABLE      blas_src_info:       NOT AVAILABLE        NOT AVAILABLE      lapack_opt_info:     lapack_mkl_info:     mkl_info:       libraries mkl,vml,guide not found in c:\Users\fname.lname\Documents\summary_python\lib       libraries mkl,vml,guide not found in C:\       NOT AVAILABLE        NOT AVAILABLE      atlas_threads_info:     Setting PTATLAS=ATLAS       libraries ptf77blas,ptcblas,atlas not found in c:\Users\fname.lname\Documents\summary_python\lib       libraries lapack_atlas not found in c:\Users\fname.lname\Documents\summary_python\lib       libraries ptf77blas,ptcblas,atlas not found in C:\       libraries lapack_atlas not found in C:\     numpy.distutils.system_info.atlas_threads_info       NOT AVAILABLE      atlas_info:       libraries f77blas,cblas,atlas not found in c:\Users\fname.lname\Documents\summary_python\lib       libraries lapack_atlas not found in c:\Users\fname.lname\Documents\summary_python\lib       libraries f77blas,cblas,atlas not found in C:\       libraries lapack_atlas not found in C:\     numpy.distutils.system_info.atlas_info       NOT AVAILABLE      lapack_info:       libraries lapack not found in c:\Users\fname.lname\Documents\summary_python\lib       libraries lapack not found in C:\       NOT AVAILABLE      lapack_src_info:       NOT AVAILABLE        NOT AVAILABLE      running egg_info     running build_src     build_src     building py_modules sources     building library "npymath" sources     No module named msvccompiler in numpy.distutils; trying from distutils     Running from numpy source directory.c:\Users\fname.lname\Documents\summary_python\build\numpy\numpy\distutils\system_info.py:531: UserWarning: Specified path  is invalid.       warnings.warn('Specified path %s is invalid.' % d)     c:\Users\fname.lname\Documents\summary_python\build\numpy\numpy\distutils\system_info.py:1417: UserWarning:         Atlas (http://math-atlas.sourceforge.net/) libraries not found.         Directories to search for the libraries can be specified in the         numpy/distutils/site.cfg file (section [atlas]) or by setting         the ATLAS environment variable.       warnings.warn(AtlasNotFoundError.__doc__)     c:\Users\fname.lname\Documents\summary_python\build\numpy\numpy\distutils\system_info.py:1426: UserWarning:         Blas (http://www.netlib.org/blas/) libraries not found.         Directories to search for the libraries can be specified in the         numpy/distutils/site.cfg file (section [blas]) or by setting         the BLAS environment variable.       warnings.warn(BlasNotFoundError.__doc__)     c:\Users\fname.lname\Documents\summary_python\build\numpy\numpy\distutils\system_info.py:1429: UserWarning:         Blas (http://www.netlib.org/blas/) sources not found.         Directories to search for the sources can be specified in the         numpy/distutils/site.cfg file (section [blas_src]) or by setting         the BLAS_SRC environment variable.       warnings.warn(BlasSrcNotFoundError.__doc__)     c:\Users\fname.lname\Documents\summary_python\build\numpy\numpy\distutils\system_info.py:1333: UserWarning:         Atlas (http://math-atlas.sourceforge.net/) libraries not found.         Directories to search for the libraries can be specified in the         numpy/distutils/site.cfg file (section [atlas]) or by setting         the ATLAS environment variable.       warnings.warn(AtlasNotFoundError.__doc__)     c:\Users\fname.lname\Documents\summary_python\build\numpy\numpy\distutils\system_info.py:1344: UserWarning:         Lapack (http://www.netlib.org/lapack/) libraries not found.         Directories to search for the libraries can be specified in the         numpy/distutils/site.cfg file (section [lapack]) or by setting         the LAPACK environment variable.       warnings.warn(LapackNotFoundError.__doc__)     c:\Users\fname.lname\Documents\summary_python\build\numpy\numpy\distutils\system_info.py:1347: UserWarning:         Lapack (http://www.netlib.org/lapack/) sources not found.         Directories to search for the sources can be specified in the         numpy/distutils/site.cfg file (section [lapack_src]) or by setting         the LAPACK_SRC environment variable.       warnings.warn(LapackSrcNotFoundError.__doc__)     error: Unable to find vcvarsall.bat     Complete output from command python setup.py egg_info:     non-existing path in 'numpy\\distutils': 'site.cfg'  F2PY Version 2  blas_opt_info:  blas_mkl_info:    libraries mkl,vml,guide not found in c:\Users\fname.lname\Documents\summary_python\lib    libraries mkl,vml,guide not found in C:\    NOT AVAILABLE    atlas_blas_threads_info:  Setting PTATLAS=ATLAS    libraries ptf77blas,ptcblas,atlas not found in c:\Users\fname.lname\Documents\summary_python\lib    libraries ptf77blas,ptcblas,atlas not found in C:\    NOT AVAILABLE    atlas_blas_info:    libraries f77blas,cblas,atlas not found in c:\Users\fname.lname\Documents\summary_python\lib    libraries f77blas,cblas,atlas not found in C:\    NOT AVAILABLE    blas_info:    libraries blas not found in c:\Users\fname.lname\Documents\summary_python\lib    libraries blas not found in C:\    NOT AVAILABLE    blas_src_info:    NOT AVAILABLE      NOT AVAILABLE    lapack_opt_info:  lapack_mkl_info:  mkl_info:    libraries mkl,vml,guide not found in c:\Users\fname.lname\Documents\summary_python\lib    libraries mkl,vml,guide not found in C:\    NOT AVAILABLE      NOT AVAILABLE    atlas_threads_info:  Setting PTATLAS=ATLAS    libraries ptf77blas,ptcblas,atlas not found in c:\Users\fname.lname\Documents\summary_python\lib    libraries lapack_atlas not found in c:\Users\fname.lname\Documents\summary_python\lib    libraries ptf77blas,ptcblas,atlas not found in C:\    libraries lapack_atlas not found in C:\  numpy.distutils.system_info.atlas_threads_info    NOT AVAILABLE    atlas_info:    libraries f77blas,cblas,atlas not found in c:\Users\fname.lname\Documents\summary_python\lib    libraries lapack_atlas not found in c:\Users\fname.lname\Documents\summary_python\lib    libraries f77blas,cblas,atlas not found in C:\    libraries lapack_atlas not found in C:\  numpy.distutils.system_info.atlas_info    NOT AVAILABLE    lapack_info:    libraries lapack not found in c:\Users\fname.lname\Documents\summary_python\lib    libraries lapack not found in C:\    NOT AVAILABLE    lapack_src_info:    NOT AVAILABLE      NOT AVAILABLE    running egg_info  running build_src  build_src  building py_modules sources  building library "npymath" sources  No module named msvccompiler in numpy.distutils; trying from distutils  Running from numpy source directory.c:\Users\fname.lname\Documents\summary_python\build\numpy\numpy\distutils\system_info.py:531: UserWarning: Specified path  is invalid.    warnings.warn('Specified path %s is invalid.' % d)  c:\Users\fname.lname\Documents\summary_python\build\numpy\numpy\distutils\system_info.py:1417: UserWarning:      Atlas (http://math-atlas.sourceforge.net/) libraries not found.      Directories to search for the libraries can be specified in the      numpy/distutils/site.cfg file (section [atlas]) or by setting      the ATLAS environment variable.    warnings.warn(AtlasNotFoundError.__doc__)  c:\Users\fname.lname\Documents\summary_python\build\numpy\numpy\distutils\system_info.py:1426: UserWarning:      Blas (http://www.netlib.org/blas/) libraries not found.      Directories to search for the libraries can be specified in the      numpy/distutils/site.cfg file (section [blas]) or by setting      the BLAS environment variable.    warnings.warn(BlasNotFoundError.__doc__)  c:\Users\fname.lname\Documents\summary_python\build\numpy\numpy\distutils\system_info.py:1429: UserWarning:      Blas (http://www.netlib.org/blas/) sources not found.      Directories to search for the sources can be specified in the      numpy/distutils/site.cfg file (section [blas_src]) or by setting      the BLAS_SRC environment variable.    warnings.warn(BlasSrcNotFoundError.__doc__)  c:\Users\fname.lname\Documents\summary_python\build\numpy\numpy\distutils\system_info.py:1333: UserWarning:      Atlas (http://math-atlas.sourceforge.net/) libraries not found.      Directories to search for the libraries can be specified in the      numpy/distutils/site.cfg file (section [atlas]) or by setting      the ATLAS environment variable.    warnings.warn(AtlasNotFoundError.__doc__)  c:\Users\fname.lname\Documents\summary_python\build\numpy\numpy\distutils\system_info.py:1344: UserWarning:      Lapack (http://www.netlib.org/lapack/) libraries not found.      Directories to search for the libraries can be specified in the      numpy/distutils/site.cfg file (section [lapack]) or by setting      the LAPACK environment variable.    warnings.warn(LapackNotFoundError.__doc__)  c:\Users\fname.lname\Documents\summary_python\build\numpy\numpy\distutils\system_info.py:1347: UserWarning:      Lapack (http://www.netlib.org/lapack/) sources not found.      Directories to search for the sources can be specified in the      numpy/distutils/site.cfg file (section [lapack_src]) or by setting      the LAPACK_SRC environment variable.    warnings.warn(LapackSrcNotFoundError.__doc__)  error: Unable to find vcvarsall.bat  ---------------------------------------- Command python setup.py egg_info failed with error code 1 Exception information: Traceback (most recent call last):   File "c:\Users\fname.lname\Documents\summary_python\lib\site-packages\pip-1.0.1-py2.7.egg\pip\basecommand.py", line 126, in main     self.run(options, args)   File "c:\Users\fname.lname\Documents\summary_python\lib\site-packages\pip-1.0.1-py2.7.egg\pip\commands\install.py", line 223, in run     requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle)   File "c:\Users\fname.lname\Documents\summary_python\lib\site-packages\pip-1.0.1-py2.7.egg\pip\req.py", line 986, in prepare_files     req_to_install.run_egg_info()   File "c:\Users\fname.lname\Documents\summary_python\lib\site-packages\pip-1.0.1-py2.7.egg\pip\req.py", line 222, in run_egg_info     command_desc='python setup.py egg_info')   File "c:\Users\fname.lname\Documents\summary_python\lib\site-packages\pip-1.0.1-py2.7.egg\pip\__init__.py", line 255, in call_subprocess     % (command_desc, proc.returncode)) InstallationError: Command python setup.py egg_info failed with error code 1 

Answer 1

I've had success installing NumPy binaries into a virtualenv with good 'ol easy_install and a little bit of un-archiving magic.

The numpy-1.x.x-win32-superpack-python2.x.exe release you download from SourceForge is really just a thin wrapper around three separate binary distributions (with SSE3, SSE2, or no SSE enabled, depending on the capabilities of your CPU). If you open up the superpack EXE file in 7-Zip (or another archive utility), you can extract those individual setup files somewhere to use separately.

Then, activate your virtual environment, and run

easy_install c:\path\to\extracted\numpy-1.x.x-sse3.exe 

to install the SSE3-optimized binaries, for example. easy_install is smart enough to find everything it needs inside that wininst bundle and will extract the compiled egg into your virtualenv's site-packages folder. I can also confirm that pip is still able to recognize and/or uninstall NumPy when you do this, and that using pip to install other packages which depend on NumPy works just fine.

The only catch is knowing which optimization level to use (SSE3, SSE2, or no-SSE). If you have a reasonably modern processor (newer than, say, a Pentium 4 or Athlon 64), it's probably safe to go with the full SSE3. You can probably also run the test suite to confirm everything works as expected.

---

I've found the easy_install "trick" to be really useful for installing all sorts of binary packages into a virtualenv. Even though I have all the requisite compilers set up on my machine, it's usually easier/faster/safer to stick with the official release when one is provided.