Performance of Tensorflow vs Tensorflow Lite

Question

Is there a performance loss when converting TensorFlow models to the TensorFlow Lite format? Because I got these results from different edge-devices:

Does it make sense that the Nvidia Jetson has a higher accuracy with the TensorFlow model (TensorRT optimized) when comparing it to the Raspberry one which is a TensorFlow Lite model.

Answer 1

Normally, there is a performance loss, but not such a significant one, more precisely around 3% in accuracy for instance in some certain models, but you have to test it on your own to check the accuracy.

Models which are subjected to TensorRT or TensorFlow-Lite do not go through the same exact conversion steps(otherwise they would be the same). Therefore, it is evident that a difference is noticeable.

To conclude: The gain in speed as compared to the performance loss(max 3%) is much more important. For each and every assumption tests should be employed.

This article is also a good read: https://www.hackster.io/news/benchmarking-tensorflow-and-tensorflow-lite-on-the-raspberry-pi-43f51b796796