Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

sklearn random forest not parallelizing

Tags:

scikit-learn

I'm using sklearn 0.16 on Ubuntu 12.04 and running:

from sklearn.ensemble import RandomForestClassifier
import numpy as np
X=np.random.rand(5000,500)
y=(np.random.rand(5000).round())
RandomForestClassifier(n_jobs=10,n_estimators=1000).fit(X,y)

However it's not using up my cores, and takes the same time as n_jobs=1. Any ideas on how to debug what's going on here?

This screenshot shows some other things running that are busy, but htop has always been showing available CPUs:

enter image description here

like image 877
Yang Avatar asked Aug 15 '26 23:08

Yang


1 Answers

try this :

import affinity
import multiprocessing
affinity.set_process_affinity_mask(0, 2**multiprocessing.cpu_count()-1)
like image 162
Kevin Roy Avatar answered Aug 19 '26 07:08

Kevin Roy