Removing features with low variance using scikit-learn

Question

scikit-learn provides various methods to remove descriptors, a basic method for this purpose has been provided by the given tutorial below,

http://scikit-learn.org/stable/modules/feature_selection.html

but the tutorial does not provide any method or a way that can tell you the way to keep the list of features that either removed or kept.

The code below has been taken from the tutorial.

    from sklearn.feature_selection import VarianceThreshold
    X = [[0, 0, 1], [0, 1, 0], [1, 0, 0], [0, 1, 1], [0, 1, 0], [0, 1, 1]]
    sel = VarianceThreshold(threshold=(.8 * (1 - .8)))
    sel.fit_transform(X)
array([[0, 1],
       [1, 0],
       [0, 0],
       [1, 1],
       [1, 0],
       [1, 1]])

The given example code above depicts only two descriptors "shape(6, 2)", but in my case, I have a huge data frames with a shape of (rows 51, columns 9000). After finding a suitable model I want to keep the track of useful and useless features because I can save computational time during the computation of the features of test data set by calculating only useful features.

For example, when you perform machine learning modeling with WEKA 6.0, it provided with remarkable flexibility over feature selection and after removing the useless feature you can get a list of a discarded features along with the useful features.

thanks

Answer 1

Then, what you can do, if I'm not wrong is:

In the case of the VarianceThreshold, you can call the method fit instead of fit_transform. This will fit data, and the resulting variances will be stored in vt.variances_ (assuming vt is your object).

Having a threhold, you can extract the features of the transformation as fit_transform would do:

X[:, vt.variances_ > threshold]

Or get the indexes as:

idx = np.where(vt.variances_ > threshold)[0]

Or as a mask

mask = vt.variances_ > threshold

PS: default threshold is 0

EDIT:

A more straight forward to do, is by using the method get_support of the class VarianceThreshold. From the documentation:

get_support([indices])  Get a mask, or integer index, of the features selected

You should call this method after fit or fit_transform.

Answer 2

this worked for me if you want to see exactly which columns are remained after thresholding you may use this method:

from sklearn.feature_selection import VarianceThreshold
threshold_n=0.95
sel = VarianceThreshold(threshold=(threshold_n* (1 - threshold_n) ))
sel_var=sel.fit_transform(data)
data[data.columns[sel.get_support(indices=True)]]