Correct Ranking Loss Implementation

Question

I have a multi-label problem and I am trying to implement the Ranking Loss as a custom loss in TensorFlow. (https://arxiv.org/pdf/1312.4894.pdf)

I made a simple CNN with a final Sigmoid layer of activations, to have independent distributions for each class.
The mathematical formulation splits the labels into two sets, positive and negative ones.

My question is, what's the correct way of implementing it?

def ranking_loss(y_true, y_pred):    
    pos = tf.where(tf.equal(y_true, 1), y_pred, tf.zeros_like(y_pred))
    neg = tf.where(tf.equal(y_true, 0), y_pred, tf.zeros_like(y_pred))

    loss = tf.maximum(1.0 - tf.math.reduce_sum(pos) + tf.math.reduce_sum(neg), 0.0)
    return tf.math.reduce_sum(loss)

The result is that for each sample, the activations scores from the positive and negative classes are summed independently.

tr = [1, 0, 0, 1]
pr = [0, 0.6, 0.55, 0.9]
t =  tf.constant([tr])
p =  tf.constant([pr])

print(ranking_loss(t, p))

tf.Tensor([[0.  0.  0.  0.9]], shape=(1, 4), dtype=float32) #Pos
tf.Tensor([[0.   0.6  0.55 0.  ]], shape=(1, 4), dtype=float32) #Neg
tf.Tensor(1.2500001, shape=(), dtype=float32) #loss

The CNN has really poor precision, recall and F1 performances.
Switching instead to a standard Binary Cross-Entropy loss result in good performances, making me think that there's something wrong in my implementation.

Answer 1

I believe that the expansion of the sums is incorrect according to the formula and that the sums tf.math.reduce_sum(pos) and tf.math.reduce_sum(neg) cannot be pushed into tf.maximum. As I see it, the formula for your example would be expanded as:

max(0, 1-0+0.6) + max(0, 1-0+0.55) + max(0, 1-0.9+0.6) + max(0, 1-0.9+0.55) = 4.5

The second implementation that you provided in the comments section looks sensible to me and yields the result that I expected. However, let me provide an alternative:

def ranking_loss(y_true, y_pred):
    y_true_ = tf.cast(y_true, tf.float32)
    partial_losses = tf.maximum(0.0, 1 - y_pred[:, None, :] + y_pred[:, :, None])
    loss = partial_losses * y_true_[:, None, :] * (1 - y_true_[:, :, None])
    return tf.reduce_sum(loss)

This implementation is probably faster.