Remove a specific feature in scikit learn

Question

Is there any way to remove a specific feature out of a scikit.learn dataset? For example, I know it is possible to remove features using sklearn.feature_selection, but these are all automated procedures removing features they decide are useless. Is there any way to implement a custom feature removal algorithm without going into the dirty insides of the data? For example, say I have a function that scores features, a toy example provided here:

def score(feature_index):
    return 0 if feature_index == 1 else 1

Now say I want to remove all those features in the iris dataset that score less than 0.5. I want to do something like this:

from sklearn import datasets
iris = datasets.load_iris()
#this is the function I want:
iris.filter_features(score, threshold=0.5)

after which I would like the iris dataset to have one less feature. Right now, I can do it like so:

from sklearn import datasets
iris = datasets.load_iris()
for feature_index in range(len(iris.feature_names)):
    if score(feature_index) < 0.5:
        iris.feature_names.pop(feature_index)
        iris.data = np.delete(iris.data, feature_index, 1)

but this looks... dirty.

Answer 1

There is no such think as a scikit-learn dataset. scikit-learn uses common datastructures are just numpy arrays (or scipy sparse matrices):

>>> from sklearn.datasets import load_iris
>>> iris = load_iris
>>> type(iris.data)
<class 'numpy.ndarray'>

You can use regular numpy array indexing to generate a new version of data. For instance to drop the second feature with boolean mask:

>>> import numpy as np
>>> X = iris.data
>>> mask = np.array([True, False, True, True])
>>> X_masked = X[:, mask]

Note, the : sign in the first location means "all the rows".

To check, you can print the first 5 rows of each array:

>>> print(X[:5])
[[ 5.1  3.5  1.4  0.2]
 [ 4.9  3.   1.4  0.2]
 [ 4.7  3.2  1.3  0.2]
 [ 4.6  3.1  1.5  0.2]
 [ 5.   3.6  1.4  0.2]]
>>> print(X_masked[:5])
[[ 5.1  1.4  0.2]
 [ 4.9  1.4  0.2]
 [ 4.7  1.3  0.2]
 [ 4.6  1.5  0.2]
 [ 5.   1.4  0.2]]

You can also use integer based fancy indexing to get the same result:

>>> index = np.array([0, 2, 3])
>>> X_indexed = X[:, index]
>>> print(X_indexed[:5])
[[ 5.1  1.4  0.2]
 [ 4.9  1.4  0.2]
 [ 4.7  1.3  0.2]
 [ 4.6  1.5  0.2]
 [ 5.   1.4  0.2]]

To learn more about basic numpy operations, have a look at a tutorial such as:

http://scipy-lectures.github.io/