Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Multiple accuracy layers in Caffe

I'm trying to classify a large set of images using nVidia DIGITS and Caffe. Everything works well when I use standard networks and networks I've constructed.

However, when I run GoogleNet example, I can see several accuracy layers' results. How can there be multiple accuracy layers in a CNN? Having multiple loss layers is quite understandable, but what do multiple accuracy values mean? I get several accuracy graphs during learning. Similar to this picture: learning process

The lossX-top1 and lossX-top5 denote accuracy layers. I understand from the prototxt that these evaluate top 1 and top 5 accuracy values, but what are the lossX accuracy layers?

Even though some of these graphs converge to around 98%, when I manually test the trained network on the 'validation.txt', I get significantly lower value (ones corresponding to the lower three accuracy graphs).

Can someone shed some light on this? How can there be multiple accuracy layers with different values?

like image 575
Igor Ševo Avatar asked Oct 06 '15 09:10

Igor Ševo


1 Answers

If you look closely at the 'train_val.prototxt' you will notice that there are indeed several accuracy layers branching off the main "path" at different levels. loss1 is evaluated after inception 4a layer, loss2 is evaluated after inception 4d and loss3 is the loss at the top of the net. Introducing loss (and accuracy layers) to intermediate representations of the deep net allows for faster gradient propagation during training. These intermediate accuracies measure how well the intermediate representations have converged.

like image 91
Shai Avatar answered Oct 06 '22 01:10

Shai