how loss and metric are calculated in cntk

Question

I am new to cntk and python. I have created a python program based on TrainResNet_CIFAR10.py to train 4736 of (64x64x3) images and test 2180 images with 4 classes. After train 160 epochs, I got loss = 0.663 and metric = 29.37%. Finished evaluation metric = 18.94%. When I evaluate the train model based on CNTKLibraryCSEvalExamples.cs to test 2180 images, almost all 2180 are classified as one class (second class). My questions are:

1. I assume loss is calculated from cross_entropy_with_softmax(z, label_var) and metric is using classification_error(z, label_var). Am I correct and how are they actually determined?
2. What does mean of metric = 29.37% and evaluation metric = 18.94%? Are they from train and test images, respectively?
3. what could cause totally wrong evaluate results?

Any help will be greatly appreciated.

Answer 1

(1) Yes.

(2) 29.37% mean that 29.37% of the classification are correct. Evaluation is on the test data, assuming you are reading both training and test data.

(3) Make sure that the input is the same format, by that I mean do you normalize or subtract the mean in your python, if so then you need to do the same in C#. Can you run the eval first using Python and see what result do you get?