How to Fine tune existing Tensorflow Object Detection model to recognize additional classes? [closed]

Question

Thanks to Google for providing a few pre-trained models with tensorflow API.

I would like to know how to retrain a pre-trained model available from the above repository, by adding new classes to the model. For example, the trained COCO dataset model has 90 classes, I would like to add 1 or 2 classes to the existing one and get one 92 class object detection model as a result.

Running Locally is provided by the repository but it is completely replacing those pre-trained classes with newly trained classes. There, only train and eval are mentioned.

So, is there any other way to retrain the model and get 92 classes as a result?

Answer 1

Question : How do we add a few more classes to my already trained network?

Specifically, we want to keep all the network as-is other than the output of the new classes. This means that for something like ResNet, we want to keep everything other than the last layer frozen, and somehow expand the last layer to have our new classes.

Answer : Combine the existing last layer with a new one you train

Specifically, we will replace the last layer with a fully connected layer that is large enough for your new classes and the old ones. Initialize it with random weights and then train it on your classes and just a few of the others. After training, copy the original weights of the original last fully connected layer into your new trained fully connected layer.

If, for example, the previous last layer was a 1024x90 matrix, and your new last layer is a 1024x92 matrix, copy the 1024x90 into the corresponding space in your new 1024x92. This will destructively replace all your training of the old classes with the pre-trained values but leave your training of your new classes. That is good, because you probably didn't train it with the same number of old classes. Do the same thing with the bias, if any.

Your final network will have only 1024x2 new weight values (plus any bias), corresponding to your new classes.

A word of caution, although this will train fast and provide quick results, it will not perform as well as retraining on a full and comprehensive data set.

That said, it'll still work well ;)

Here is a reference to how to replace the last layer How to remove the last layer from trained model in Tensorflow that someone else answered