Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Make sure numpy is using MKL library on mac pro

Tags:

I am using Enthought's Canopy/EPD version of python which ships with numpy linked against MKL. I am currently running a set of simulations in parallel (using PP) on both my work computer (Windows 7, Quad Core i5 @ 3.33 Ghz, 4 GB ram) and my home workstation (Mac Pro 3.1, Ubuntu 12.04, 2x Quad Core Xeon @ 2.8 Ghz, 6 GB ram).

But when I benchmark my simulations, they run much quicker on the work computer (35 seconds per iteration vs. 60 on the mac pro). The problem is being perfectly balanced between cores (embarrassingly parallel problem), so I suspect there is an issue with the MKL library on the linux workstation at home. Is there a way to verify that the MKL library is actually being used in python. I've read threads that say you can check to see if python is linked to it, but it doesn't ensure that it was built correctly and is actually being used.

like image 437
user1554752 Avatar asked Mar 25 '14 20:03

user1554752


People also ask

Is NumPy using MKL?

NumPy doesn't depend on any other Python packages, however, it does depend on an accelerated linear algebra library - typically Intel MKL or OpenBLAS. Users don't have to worry about installing those (they're automatically included in all NumPy install methods).

What is NumPy MKL?

In particular, Intel has one called "MKL" - Intel Math Kernel Library. So numpy-mkl just means a version of numpy compiled against the MKL fortran library. Probably whichever version of numpy you previously had installed was somehow broken, and couldn't find the libraries it needed.

Does SciPy use MKL?

On other Intel-based systems, there are several freely available Python distributions with versions of NumPy and SciPy that are linked against the Intel Math Kernel Library (MKL).


1 Answers

>>> numpy.show_config() 

You will see output something like this, showing that MKL is indeed linked.

lapack_opt_info:     libraries = ['mkl_lapack95_lp64', 'mkl_intel_lp64', 'mkl_intel_thread', 'mkl_core', 'mkl_mc', 'mkl_mc3', 'pthread']     library_dirs = ['/Users/vagrant/src/master-env/Resources/Python.app/Contents/MacOS/../../../../lib']     define_macros = [('SCIPY_MKL_H', None)]     include_dirs = ['/Users/vagrant/src/master-env/Resources/Python.app/Contents/MacOS/../../../../include'] blas_opt_info:     libraries = ['mkl_intel_lp64', 'mkl_intel_thread', 'mkl_core', 'mkl_mc', 'mkl_mc3', 'pthread']     library_dirs = ['/Users/vagrant/src/master-env/Resources/Python.app/Contents/MacOS/../../../../lib']     define_macros = [('SCIPY_MKL_H', None)]     include_dirs = ['/Users/vagrant/src/master-env/Resources/Python.app/Contents/MacOS/../../../../include'] openblas_info: NOT AVAILABLE lapack_mkl_info:     libraries = ['mkl_lapack95_lp64', 'mkl_intel_lp64', 'mkl_intel_thread', 'mkl_core', 'mkl_mc', 'mkl_mc3', 'pthread']     library_dirs = ['/Users/vagrant/src/master-env/Resources/Python.app/Contents/MacOS/../../../../lib']     define_macros = [('SCIPY_MKL_H', None)]     include_dirs = ['/Users/vagrant/src/master-env/Resources/Python.app/Contents/MacOS/../../../../include'] blas_mkl_info:     libraries = ['mkl_intel_lp64', 'mkl_intel_thread', 'mkl_core', 'mkl_mc', 'mkl_mc3', 'pthread']     library_dirs = ['/Users/vagrant/src/master-env/Resources/Python.app/Contents/MacOS/../../../../lib']     define_macros = [('SCIPY_MKL_H', None)]     include_dirs = ['/Users/vagrant/src/master-env/Resources/Python.app/Contents/MacOS/../../../../include'] mkl_info:     libraries = ['mkl_intel_lp64', 'mkl_intel_thread', 'mkl_core', 'mkl_mc', 'mkl_mc3', 'pthread']     library_dirs = ['/Users/vagrant/src/master-env/Resources/Python.app/Contents/MacOS/../../../../lib']     define_macros = [('SCIPY_MKL_H', None)]     include_dirs = ['/Users/vagrant/src/master-env/Resources/Python.app/Contents/MacOS/../../../../include'] 
like image 155
Jonathan March Avatar answered Sep 18 '22 06:09

Jonathan March