Getting a Scoring Function by Name in scikit-learn

Question

In scikit-learn , there is the notion of a scoring function. If we have some predicted labels and the true labels, we can get to the score by calling scoring(y_true, y_predict). An example of such scoring function is sklearn.metrics.accuracy_score.

A scoring function is not to be confused of the scorer, which is an object that can be called as scorer(estimator, X, y_true).

There are many builtin scorers in scikit-learn. It is possible to get to these scorers by their string names. For example, we can get the scorer corresponding to the name 'accuracy' by calling sklearn.metrics.get_scorer("accuracy")/

But it turns out that there is no obvious mechanism to access the built-in scoring functions by their names at run-time, through passing in the name as a string. For example, there is no way to access sklearn.metrics.accuracy_score by its name accuracy.

For example, if at run time, the program knows the name of the scoring function is contained in variable name, I am looking for a mechanism get_scoring_function(), such that, get_scoring_function(name) will return the scoring function handle. Note that this name, name, is not known at scripting time.

Is there any way to access the built-in scoring functions by their names at run time through passing in the names as strings?

Answer 1

You can use the get_scorer() function, which accepts a string as an argument, and then get the _score_func attribute of the returned object.

So for example

from sklearn.metrics import get_scorer

get_scorer('accuracy')._score_func(y_true, y_pred) 

is equivalent to

from sklearn.metrics import accuracy_score

accuracy_score(y_true, y_pred)