Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to avoid this four-line memory leak with NumPy+MKL?

The following simple four-line code produces a memory leak in my Python 2.6.6 / NumPy 1.7.0 / MKL 10.3.6 setup:

import numpy as np

t = np.random.rand(10,10)
while True:
  t = t / np.trace(t)

With each operation, the used memory grows by the size of a 10x10 matrix. However, there is no such behaviour when I use a NumPy 1.4.1/ATLAS setup.

I have read about MKL not necessarily freeing memory automatically, so I guess this is the reason for the blowup. Is there a simple way to modify NumPy (before or after compilation), such that this four-liner will work fine?

Output of np.show_config()

numpy 1.7.0

lapack_opt_info:
    libraries = ['mkl_rt', 'pthread']
    library_dirs = ['$MKLPATH/lib/intel64']
    define_macros = [('SCIPY_MKL_H', None)]
    include_dirs = ['$MKLPATH/include']
blas_opt_info:
    libraries = ['mkl_rt', 'pthread']
    library_dirs = ['$MKLPATH/lib/intel64']
    define_macros = [('SCIPY_MKL_H', None)]
    include_dirs = ['$MKLPATH/include']
lapack_mkl_info:
    libraries = ['mkl_rt', 'pthread']
    library_dirs = ['$MKLPATH/lib/intel64']
    define_macros = [('SCIPY_MKL_H', None)]
    include_dirs = ['$MKLPATH/include']
blas_mkl_info:
    libraries = ['mkl_rt', 'pthread']
    library_dirs = ['$MKLPATH/lib/intel64']
    define_macros = [('SCIPY_MKL_H', None)]
    include_dirs = ['$MKLPATH/include']
mkl_info:
    libraries = ['mkl_rt', 'pthread']
    library_dirs = ['$MKLPATH/lib/intel64']
    define_macros = [('SCIPY_MKL_H', None)]
    include_dirs = ['$MKLPATH/include']
like image 316
cm_ Avatar asked Mar 03 '13 21:03

cm_


1 Answers

This is indeed a NumPy bug, which has been known for some months and has been discussed here; it will be fixed in 1.7.1. The fix is this nice one-liner in item_selection.c. After adding this line and recompiling, everything works fine.

like image 167
cm_ Avatar answered Sep 20 '22 00:09

cm_