the loss of mse always be 0 when keras for topic predict

Question

my input is a 200 dims vector, which is generated by mean of the word2vector of all words of a article, my output is a 50 dims vector,which is generated by the LDA results of a article I want to use mse as the loss function,but the value of the loss always be 0 my code as follows:

<pre>model = Sequential()
model.add(Dense(cols*footsize, 400,init = "glorot_uniform"))
# model.add(LeakyReLU(alpha = 0.3))
model.add(Activation('relu'))
model.add(Dropout(0.2))
model.add(Dense(400, 400,init = "glorot_uniform"))
model.add(Activation('relu'))
model.add(Dropout(0.2))
model.add(Dense(400, 50,init = "glorot_uniform"))
model.add(Activation('softmax'))
model.compile(loss='mse', optimizer='rmsprop')</pre>

the screen output as follows:

who can tell me why ,thanks!

Answer 1

First, is your output a one-hot vector of predicted classes? IE: class one is [1, 0, 0, ...] and class two is [0, 1, 0, 0, ...].

If so, then using softmax activation at the output layer is acceptable and you are doing a classification problem. If you are doing a classification problem (one-hot output) you cannot use MSE as the loss, use categorical cross-entropy.

Softmax scales the output so that the number given is a predicted probability of a certain class. Wikipedia here: https://en.wikipedia.org/wiki/Softmax_function

If you are expecting the output vector to be real numbers then you need to use linear activation on your output neurons.