Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Compile numpy WITHOUT Intel MKL/BLAS/ATLAS/LAPACK

I am using py2exe to convert a script which uses numpy and am getting a very large resulting folder, and it seems a lot of the large files are from parts of the numpy package that I'm not using, such as numpy.linalg.

To reduce the size of folder that is created, I have been led to believe I should have numpy compiled without Intel MKL/BLAS/ATLAS/LAPACK.

How would I make this change?

EDIT
In C:\Python27\Lib\site-packages\numpy\linalg I found the following files: _umath_linalg.pyd (34MB) and lapack_lite.pyd (18MB) which are being copied into the distribution folder when using py2exe. If possible I would like to remove dependence on these while still being able to use numpy arrays. The other large file being included is in C:\Python27\Lib\site-packages\numpy\core and is called _dotblas.pyd (12MB). Is it possible to remove this too?

like image 565
Siwel Avatar asked Aug 26 '15 14:08

Siwel


1 Answers

According to the official documentation:

Disabling ATLAS and other accelerated libraries

Usage of ATLAS and other accelerated libraries in Numpy can be disabled via:

  BLAS=None LAPACK=None ATLAS=None python setup.py build

However, this information seems to be out of date, since I found that even with these options numpy v1.9.2 was still automatically finding libopenblas.so:

numpy_source_dir/$ BLAS=None LAPACK=None ATLAS=None python setup.py config
...
openblas_info:
  FOUND:
    libraries = ['openblas', 'openblas']
    library_dirs = ['/opt/OpenBLAS/lib']
    language = f77

  FOUND:
    libraries = ['openblas', 'openblas']
    library_dirs = ['/opt/OpenBLAS/lib']
    language = f77
...

One workaround is to copy site.cfg.example to site.cfg, then edit it to make the paths to the relevant BLAS/LAPACK libraries invalid:

[openblas]
libraries =
library_dirs =
include_dirs =

When you subsequently call BLAS=None LAPACK=None ATLAS=None python setup.py config you should get an output containing this:

...
openblas_info:
/home/alistair/src/python/numpy/numpy/distutils/system_info.py:594: UserWarning: Specified path  is invalid.
  warnings.warn('Specified path %s is invalid.' % d)
  libraries  not found in []
  NOT AVAILABLE
...

I expect that the same approach will work for ATLAS and MKL, although I don't have these libraries installed in order to do a proper test.

You should, of course, be aware that not having accelerated BLAS/LAPACK libraries will have a big detrimental effect on performance for linear algebra ops.


Update

As mentioned in the comments below, you didn't actually "compile" your current version of numpy, but rather installed it from a binary distribution. The approach I gave above would require you to build numpy from source, which is not an easy thing to do in Windows (although there are official instructions here).

A much easier option would be to install one of the unoptimized numpy binaries available from Christoph Gohlke's website here.

like image 79
ali_m Avatar answered Sep 30 '22 08:09

ali_m