Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Class weights in binary classification model with Keras

We know that we can pass a class weights dictionary in the fit method for imbalanced data in binary classification model. My question is that, when using only 1 node in the output layer with sigmoid activation, can we still apply the class weights during the training?

model = Sequential()
model.add(Dense(64, activation='tanh',input_shape=(len(x_train[0]),)))
model.add(Dense(1, activation='sigmoid')) 

model.compile(
    optimizer=optimizer, 
    loss=loss, 
    metrics=metrics)

model.fit(
    x_train, y_train, 
    epochs=args.e, 
    batch_size=batch_size,
    class_weight={0: 1, 1: 3})
like image 387
Jie HE Avatar asked Apr 13 '17 10:04

Jie HE


People also ask

How do I determine my class weight?

Generating class weights In binary classification, class weights could be represented just by calculating the frequency of the positive and negative class and then inverting it so that when multiplied to the class loss, the underrepresented class has a much higher error than the majority class.

How do class weights work?

Class weights give all the classes equal importance on gradient updates, on average, regardless of how many samples we have from each class in the training data. This prevents models from predicting the more frequent class more often just because it's more common.

Which Optimizer is best for binary classification?

For binary classification problems that give output in the form of probability, binary_crossentropy is usually the optimizer of choice.


1 Answers

If you want to fully control this weight, why not write a custom loss function?

from keras import backend as K
def weighted_binary_crossentropy( y_true, y_pred, weight=1. ) :
    y_true = K.clip(y_true, K.epsilon(), 1-K.epsilon())
    y_pred = K.clip(y_pred, K.epsilon(), 1-K.epsilon())
    logloss = -(y_true * K.log(y_pred) * weight + (1 - y_true) * K.log(1 - y_pred))
    return K.mean( logloss, axis=-1)
like image 111
pitfall Avatar answered Nov 02 '22 18:11

pitfall