What are the best metrics for Multi-Object Tracking (MOT) evaluation and why?

Question

I want to compare multiple computer vision Multi-Object Tracking (MOT) methods on my own dataset, so first I want to choose the best metrics for this task. I have carried out some research in scientific literature and I come to the conclusion that there are three main metrics sets:

1. Metrics from "Tracking of Multiple, Partially Occluded Humans based on Static Body Part Detection"
2. CLEAR MOT metrics
3. ID scores

Therefore, I wonder to which of the above metrics should I attach the greatest importance?

And I would like to ask if anyone has encountered a similar issue and has any thoughts on this topic that could justify and help me to choose the best metrics for the above task.

Answer 1

I know this is old but I see nobody mentioning HOTA (https://arxiv.org/pdf/2009.07736.pdf). This metric has become the new standard for multi-object tracking as can be seen in the latest SOTA tracking research: https://arxiv.org/abs/2202.13514 and https://arxiv.org/pdf/2110.06864.pdf

The reason behind using a metric that is not MOTA and IDF1 is that they overemphasize detection and association respectively. HOTA explicitly measures both types of errors and combines these in a balanced way. HOTA also incorporates measuring the localization accuracy of tracking results which isn’t present in either MOTA or IDF1.