Keras multi-label image classification with F1-score

Question

I am working on a multi-label image classification problem with the evaluation being conducted in terms of F1-score between system predicted and ground truth labels.

Given that, should I use loss="binary_crossentropy" or loss=keras_metrics.f1_score() where keras_metrics.f1_score() is taken from here: https://pypi.org/project/keras-metrics/? I am a bit confused because all of the tutorials I have found on the Internet regarding multi-label classification are based on the binary_crossentropy loss function, but here I have to optimize against F1-score.

Furthermore, should I set metrics=["accuracy"] or maybe metrics=[keras_metrics.f1_score()] or I should left this completely empty?

Answer 1

Based on user706838 answer ...

use the f1_score in https://www.kaggle.com/rejpalcz/best-loss-function-for-f1-score-metric

import tensorflow as tf
import keras.backend as K

def f1_loss(y_true, y_pred):

    tp = K.sum(K.cast(y_true*y_pred, 'float'), axis=0)
    tn = K.sum(K.cast((1-y_true)*(1-y_pred), 'float'), axis=0)
    fp = K.sum(K.cast((1-y_true)*y_pred, 'float'), axis=0)
    fn = K.sum(K.cast(y_true*(1-y_pred), 'float'), axis=0)

    p = tp / (tp + fp + K.epsilon())
    r = tp / (tp + fn + K.epsilon())

    f1 = 2*p*r / (p+r+K.epsilon())
    f1 = tf.where(tf.is_nan(f1), tf.zeros_like(f1), f1)
    return 1 - K.mean(f1)