Menu
I'm using mixture submodule of sklearn module for Gaussian Mixture Model... When I run my code on a multicore system, it uses multiple cores even though I do not ask for it in the code. Is this a default behavior? And more important, how can I disable it?
Thanks
374
Scikit-learn relies heavily on NumPy and SciPy, which internally call multi-threaded linear algebra routines implemented in libraries such as MKL, OpenBLAS or BLIS.
Python doesn't support multi-threading because Python on the Cpython interpreter does not support true multi-core execution via multithreading. However, Python does have a threading library. The GIL does not prevent threading.
This is why Python multithreading can provide a large speed increase. The processor can switch between the threads whenever one of them is ready to do some work. Using the threading module in Python or any other interpreted language with a GIL can actually result in reduced performance.
Python does not support multithreading as the CPython interpreter does not support multi-core execution through multithreading. It will not allow you to use the extra CPU cores.
If you are using MKL then try
export MKL_NUM_THREADS=1
For Numpy with OpenBLAS:
export OPENBLAS_NUM_THREADS=1
For some versions of Numpy this variation has been suggested:
export NUMEXPR_NUM_THREADS=1
The environment variable has to be set before the script is run (setting inside the script itself does not have the desired effect). For setting threads at runtime see: Set max number of threads at runtime on numpy/openblas
See the following for identifying how your numpy is setup: How to check blas/lapack linkage in numpy/scipy?
155
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
