Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Keras model.compile: metrics to be evaluated by the model

Tags:

keras

I am following some Keras tutorials and I understand the model.compile method creates a model and takes the 'metrics' parameter to define what metrics are used for evaluation during training and testing.

compile(self, optimizer, loss, metrics=[], sample_weight_mode=None)

The tutorials I follow typically use "metrics=['accuracy']". I would like to use other metrics such as fmeasure, and reading https://keras.io/metrics/ I know there is a wide range of options. but I do not know how to pass them to the compile method?

For example:

model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['fmeasure'])

will generate an error saying that there is no such metric.

Any suggestions highly appreciated

Thanks

like image 765
Ziqi Avatar asked Nov 30 '16 12:11

Ziqi


People also ask

What are metrics in model compile?

A metric is a function that is used to judge the performance of your model. Metric functions are similar to loss functions, except that the results from evaluating a metric are not used when training the model. Note that you may use any loss function as a metric.

What is the model compile () method used for in Keras?

compile method. Configures the model for training. optimizer: String (name of optimizer) or optimizer instance. See tf.


2 Answers

There are two types of metrics that you can provide.
First are the one provided by keras which you can find here which you provide in single quotes like 'mae' or also you can define like

from keras import metrics
model.compile(loss='mean_squared_error',
              optimizer='sgd',
              metrics=[metrics.mae, metrics.categorical_accuracy]) \\or like 
              metrics=['mae', 'categorical_accuracy']

Second is custom metrics like this

import keras.backend as K

def mean_pred(y_true, y_pred):
    return K.mean(y_pred)

model.compile(optimizer='rmsprop',
              loss='binary_crossentropy',
              metrics=['accuracy', mean_pred])

Here mean_pred is the custom metric. See the difference in defining the already available metrics and custom defined metrics. So fmeasure is not readily available. You have to define it as custom function.

like image 141
Sargam Modak Avatar answered Oct 05 '22 19:10

Sargam Modak


I believe that your question is similar to https://stackoverflow.com/a/43354147/6701627. Please check the answer in the given post.

PS: I intended to put this as a comment, but don't have sufficient reputation points.

like image 29
thepunitsingh Avatar answered Oct 05 '22 19:10

thepunitsingh