Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Missing values in scikits machine learning

Is it possible to have missing values in scikit-learn ? How should they be represented? I couldn't find any documentation about that.

like image 561
Vladtn Avatar asked Feb 20 '12 17:02

Vladtn


People also ask

Which machine learning algorithm can handle missing values?

k-NN and Random Forest algorithms can also support missing values. the k-NN algorithm considers the missing values by taking the majority of the K nearest values.

Does decision tree support missing values?

Decision Tree can automatically handle missing values. Decision Tree is usually robust to outliers and can handle them automatically.


1 Answers

Missing values are simply not supported in scikit-learn. There has been discussion on the mailing list about this before, but no attempt to actually write code to handle them.

Whatever you do, don't use NaN to encode missing values, since many of the algorithms refuse to handle samples containing NaNs.

The above answer is outdated; the latest release of scikit-learn has a class Imputer that does simple, per-feature missing value imputation. You can feed it arrays containing NaNs to have those replaced by the mean, median or mode of the corresponding feature.

like image 165
Fred Foo Avatar answered Oct 02 '22 10:10

Fred Foo