Dynamic quantization in Pytorch starts random training after quantization

Question

When I run following code for dynamic quantization it starts training with some random natural images for 100 epochs, I don't want to do training again. I have pretrained weights, I just want to quantize my pretrained weights to reduce inference time:

from ultralytics import YOLO
import torch
import torch.quantization

model=YOLO('pre_trained_weights.pt')

model.load_state_dict(torch.load('checkpoint.pth')) #donot know this step is necessary or not

qmodel = torch.quantization.quantize_dynamic(model, dtype = torch.quint8)

I tried above code and I am expecting that I just want to quantize my pretrained weights to reduce inference time

Answer 1

Dynamic quantization is not available for Conv layer(Conv in YOLO). Checkout the documentation here. I believe it is going into calibration mode when you apply on layers which are not supported.