Menu
I am working on a net in tensorflow which produces a vector which is then passed through a softmax which is my output.
Now I have been testing this and weirdly enough the vector (the one that passed through softmax) has zeros in all coordinate but one.
Based on the softmax's definition with the exponential, I assumed that this wasn't supposed to happen. Is this an error?
EDIT: My vector is 120x160 =192000. All values are float32
366
Softmax is an activation function that scales numbers/logits into probabilities. The output of a Softmax is a vector (say v ) with probabilities of each possible outcome. The probabilities in vector v sums to one for all possible outcomes or classes.
Softmax is a function that finds the probabilities of all the classes such that they sum to 1. Why S2 is correct ? If we are using a softmax, in order for the probability of one class to increase, the probabilities of at least one of the other classes has to decrease by an equivalent amount.
It ensures that all the output values of the function will sum to 1 and each be in the range (0, 1), thus constituting a valid probability distribution. The number of classes in the multi-class classifier.
It may not be an error. You need to look at the input to the softmax as well. It is quite possible this vector has very negative values and a single very positive value. This would result in a softmax output vector containing all zeros and a single one value.
You correctly pointed out that the softmax numerator should never have zero-values due to the exponential. However, due to floating point precision, the numerator could be a very small value, say, exp(-50000), which essentially evaluates to zero.
88
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
