We know that we can pass a class weights dictionary in the fit method for imbalanced data in binary classification model. My question is that, when using only 1 node in the output layer with sigmoid activation, can we still apply the class weights during the training?
model = Sequential()
model.add(Dense(64, activation='tanh',input_shape=(len(x_train[0]),)))
model.add(Dense(1, activation='sigmoid'))
model.compile(
optimizer=optimizer,
loss=loss,
metrics=metrics)
model.fit(
x_train, y_train,
epochs=args.e,
batch_size=batch_size,
class_weight={0: 1, 1: 3})
Generating class weights In binary classification, class weights could be represented just by calculating the frequency of the positive and negative class and then inverting it so that when multiplied to the class loss, the underrepresented class has a much higher error than the majority class.
Class weights give all the classes equal importance on gradient updates, on average, regardless of how many samples we have from each class in the training data. This prevents models from predicting the more frequent class more often just because it's more common.
For binary classification problems that give output in the form of probability, binary_crossentropy is usually the optimizer of choice.
If you want to fully control this weight, why not write a custom loss function?
from keras import backend as K
def weighted_binary_crossentropy( y_true, y_pred, weight=1. ) :
y_true = K.clip(y_true, K.epsilon(), 1-K.epsilon())
y_pred = K.clip(y_pred, K.epsilon(), 1-K.epsilon())
logloss = -(y_true * K.log(y_pred) * weight + (1 - y_true) * K.log(1 - y_pred))
return K.mean( logloss, axis=-1)
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With