PyTorch model to C++

Question

I have trained the detection algorithm and saved my best model. Now I want to convert my model (pretrained) to C++ and use it in my app. I wanted to know what are the possible ways to convert a pyTorch model to c++?

Thanks!

Answer 1

• You can use TorchScript intermediate representation of a PyTorch model, through tracing and scripting, that can be run in C++ environment. For this, you'll probably have to modify the model itself in order for it to be traced or scripted.

• You can use ONNX (Open Neural Network Exchange), through which you can export your model and load it in another C++ framework such as Caffe. It comes with its own implications though.

• The easiest is to try Embedding Python, through which you can run your python (pytorch) model in C++ environment. Note that the model will still run in python, but only through C++, so there won't be any speed gains that you might be expecting in C++.

Also, with the release of torchvision 0.5, all models in torchvision have native support for TorchScript and ONNX.