Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to calculate the mean IU score in image segmentation?

Tags:

How to compute the mean IU (mean Intersection over Union) score as in this paper?

Long, Jonathan, Evan Shelhamer, and Trevor Darrell. "Fully Convolutional Networks for Semantic Segmentation."

like image 818
hkcqr Avatar asked Jul 27 '15 12:07

hkcqr


People also ask

How IoU score is calculated?

IoU is calculated by dividing the overlap between the predicted and ground truth annotation by the union of these. It is not a problem if you are unfamiliar with the mathematical notation because the Intersection over Union formula can be easily visualized.

How do you calculate accuracy of segmentation?

So, I suggest you can use the following measures to evaluate your segmentation result: True positive rate: the correctly segmentation area over all the area you segmented. False positive rate: the area that is not in the ground truth but that is in your result over all the area you segmented. Accuracy.

What is a good IoU score in semantic segmentation?

Intersection over union (IoU) is known to be a good metric for measuring overlap between two bounding boxes or masks. If the prediction is completely correct, IoU = 1. The lower the IoU, the worse the prediction result. The proposed approach has worked well in practice.

What is a good MIoU score?

The threshold to be considered accpetable prediction is 0.5. If IoU is larger than 0.5, it is normally considered a good prediction [15, 31] .


2 Answers

For each class Intersection over Union (IU) score is:

true positive / (true positive + false positive + false negative)

The mean IU is simply the average over all classes.


Regarding the notation in the paper:

  • n_cl : the number of classes
  • t_i : the total number of pixels in class i
  • n_ij : the number of pixels of class i predicted to belong to class j. So for class i:

    • n_ii : the number of correctly classified pixels (true positives)
    • n_ij : the number of pixels wrongly classified (false positives)
    • n_ji : the number of pixels wrongly not classifed (false negatives)

You can find the matlab code to compute this directly in the Pascak DevKit here

like image 72
Miki Avatar answered Oct 21 '22 13:10

Miki


 from sklearn.metrics import confusion_matrix    import numpy as np   def compute_iou(y_pred, y_true):      # ytrue, ypred is a flatten vector      y_pred = y_pred.flatten()      y_true = y_true.flatten()      current = confusion_matrix(y_true, y_pred, labels=[0, 1])      # compute mean iou      intersection = np.diag(current)      ground_truth_set = current.sum(axis=1)      predicted_set = current.sum(axis=0)      union = ground_truth_set + predicted_set - intersection      IoU = intersection / union.astype(np.float32)      return np.mean(IoU) 
like image 32
Alex-zhai Avatar answered Oct 21 '22 12:10

Alex-zhai