Does changing a token name in an image caption model affect performance?

Question

If I train an image caption model then stop to rename a few tokens:

1. Should I train the model from scratch?
2. Or can I reload the model and continue training from the last epoch with the updated vocabulary?

Will either approach effect model accuracy/performance differently?

Answer 1

I would go for option 2.

When training the model from scratch, you are initializing the model's weights randomly and then you fit them based on your problem. However, if, instead of using random weights, you use weights that have already been trained for a similar problem, you may decrease the convergence time. This option is kind similar to the idea of transfer learning.