Test score vs test accuracy when evaluating model using Keras

Question

Im using a neural network implemented with the Keras library and below is the results during training. At the end it prints a test score and a test accuracy. I can't figure out exactly what the score represents, but the accuracy I assume to be the number of predictions that was correct when running the test.

Epoch 1/15 1200/1200 [==============================] - 4s - loss: 0.6815 - acc: 0.5550 - val_loss: 0.6120 - val_acc: 0.7525

Epoch 2/15 1200/1200 [==============================] - 3s - loss: 0.5481 - acc: 0.7250 - val_loss: 0.4645 - val_acc: 0.8025

Epoch 3/15 1200/1200 [==============================] - 3s - loss: 0.5078 - acc: 0.7558 - val_loss: 0.4354 - val_acc: 0.7975

Epoch 4/15 1200/1200 [==============================] - 3s - loss: 0.4603 - acc: 0.7875 - val_loss: 0.3978 - val_acc: 0.8350

Epoch 5/15 1200/1200 [==============================] - 3s - loss: 0.4367 - acc: 0.7992 - val_loss: 0.3809 - val_acc: 0.8300

Epoch 6/15 1200/1200 [==============================] - 3s - loss: 0.4276 - acc: 0.8017 - val_loss: 0.3884 - val_acc: 0.8350

Epoch 7/15 1200/1200 [==============================] - 3s - loss: 0.3975 - acc: 0.8167 - val_loss: 0.3666 - val_acc: 0.8400

Epoch 8/15 1200/1200 [==============================] - 3s - loss: 0.3916 - acc: 0.8183 - val_loss: 0.3753 - val_acc: 0.8450

Epoch 9/15 1200/1200 [==============================] - 3s - loss: 0.3814 - acc: 0.8233 - val_loss: 0.3505 - val_acc: 0.8475

Epoch 10/15 1200/1200 [==============================] - 3s - loss: 0.3842 - acc: 0.8342 - val_loss: 0.3672 - val_acc: 0.8450

Epoch 11/15 1200/1200 [==============================] - 3s - loss: 0.3674 - acc: 0.8375 - val_loss: 0.3383 - val_acc: 0.8525

Epoch 12/15 1200/1200 [==============================] - 3s - loss: 0.3624 - acc: 0.8367 - val_loss: 0.3423 - val_acc: 0.8650

Epoch 13/15 1200/1200 [==============================] - 3s - loss: 0.3497 - acc: 0.8475 - val_loss: 0.3069 - val_acc: 0.8825

Epoch 14/15 1200/1200 [==============================] - 3s - loss: 0.3406 - acc: 0.8500 - val_loss: 0.2993 - val_acc: 0.8775

Epoch 15/15 1200/1200 [==============================] - 3s - loss: 0.3252 - acc: 0.8600 - val_loss: 0.2960 - val_acc: 0.8775

400/400 [==============================] - 0s

Test score: 0.299598811865

Test accuracy: 0.88

Looking at the Keras documentation, I still don't understand what score is. For the evaluate function, it says:

Returns the loss value & metrics values for the model in test mode.

One thing I noticed is that when the test accuracy is lower, the score is higher, and when accuracy is higher, the score is lower.

Answer 1

For reference, the two relevant parts of the code:

model.compile(loss='binary_crossentropy',               optimizer='adam',               metrics=['accuracy'])  score, acc = model.evaluate(x_test, y_test,                             batch_size=batch_size) print('Test score:', score) print('Test accuracy:', acc) 

Score is the evaluation of the loss function for a given input.

Training a network is finding parameters that minimize a loss function (or cost function).

The cost function here is the binary_crossentropy.

For a target T and a network output O, the binary crossentropy can defined as

f(T,O) = -(T*log(O) + (1-T)*log(1-O) )

So the score you see is the evaluation of that.

If you feed it a batch of inputs it will most likely return the mean loss.

So yeah, if your model has lower loss (at test time), it should often have lower prediction error.