Menu
I was wondering if a model trained on the GPU could be use to run inference with the cpu ? (And vice versa) Thanks to you!
228
After all, GPUs substantially speed up deep learning training, and inference is just the forward pass of your neural network that's already accelerated on GPU. This is true, and GPUs are indeed an excellent hardware accelerator for inference.
CPUs are everywhere and can serve as more cost-effective options for running AI-based solutions compared to GPUs. However, finding models that are both accurate and can run efficiently on CPUs can be a challenge. Generally speaking, GPUs are 3X faster than CPUs.
You can do it as long as your model doesn't have explicit device allocations. IE, if your model has blocks like with tf.device('gpu:0'), it'll complain when you run it on model without GPU.
In such cases you must make sure your imported model doesn't have explicit device assignments, for instance, but using clear_devices argument in import_meta_graph
189
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
