Which SSIM is correct : skimage.metrics.structural_similarity()?

Question

Stackoverflow comunity,

I'm trying to compute SSIM (Structural SIMilarity) between two bmp images on Python. I've found structural_similarity() function implemented in the skimage python library and the equivalent code from the original MatLab implementation which is hosted here. The implimentation is right bellow:

def structuralSimilarityIndex(ref_image, impaired_image, cs_map=False):

    window = Metric.fSpecialGauss(constant.SSIM_FILTER_SIZE,
                                  constant.SSIM_FILTER_SIGMA)
    C1 = (constant.SSIM_Constant_1 * constant.PIXEL_MAX) ** 2
    C2 = (constant.SSIM_Constant_2 * constant.PIXEL_MAX) ** 2

    mu1 = signal.fftconvolve(window, ref_image, mode='valid')
    mu2 = signal.fftconvolve(window, impaired_image, mode='valid')

    mu1_sq = mu1 * mu1
    mu2_sq = mu2 * mu2
    mu1_mu2 = mu1 * mu2

    sigma1_sq = signal.fftconvolve(
        window, ref_image*ref_image, mode='valid') - mu1_sq
    sigma2_sq = signal.fftconvolve(
        window, impaired_image*impaired_image, mode='valid') - mu2_sq
    sigma12 = signal.fftconvolve(
        window, ref_image*impaired_image, mode='valid') - mu1_mu2

    if cs_map:
        return (((2 * mu1_mu2 + C1) * (2 * sigma12 + C2)) / ((mu1_sq + mu2_sq + C1) * (sigma1_sq + sigma2_sq + C2)), (2.0 * sigma12 + C2) / (sigma1_sq + sigma2_sq + C2))
    else:
        return np.mean(((2 * mu1_mu2 + C1) * (2 * sigma12 + C2)) / ((mu1_sq + mu2_sq + C1) * (sigma1_sq + sigma2_sq + C2)))

I'm reading the images using this piece of code :

ref_image = np.asfarray(Image.open('ref_image.bmp').convert('L'))
impaired_image = np.asfarray(Image.open('impaired_image.bmp').covert('L)

The input images shape and dtype of both ref_image and impaired_image are respectively:

(512, 512) float64

(512, 512) float64

I've tested both using the same condition and same input images as follow:

# Using the above code
structuralSimilarityIndex(ref_image, impaired_image, cs_map=False)

# Using the function imported from skimage.metrics
structural_similarity(ref_image, impaired_image, gaussian_weights=False, use_sample_covariance=False)

the result was so much different, here the results:

The SSIM from the Skimage python library:

SSIM : 0.38135154028457885

The SSIM from the code above:

SSIM : 0.8208087737160036

EDIT:

I've added the reading and calling code

The above Python code was from the signal processing library, which is according to the author, the function attempts to mimic precisely the functionality of ssim.m a MATLAB provided by the author's of SSIM

Update :

I've tested the original code which is writing in MatLab on the same images and the result is as follow :

SSIM : 0.8424

Which is not far from the result of the Python implementation given above.

Answer 1

I've opened an issue on the scikit-image Github repository and I got an answer. Here the answer, I've change nothing to the answer and you can find it here:

I think the primary issue here is that the way you computed images from PIL results in floating point images, but ones where the values are in the range [0, 255.0]. skimage will assume a range [-1.0, 1.0] for data_range when the input is floating-point, so you will need to manually specify data_range=255.

Also, see the Notes section of the docstring for recommendations to set gaussian_weights=True, sigma=1.5 to more closely match the Matlab script by Wang et. al. (I think recent Matlab also has its own built-in SSIM implementation, but I haven't tried comparing to that case and don't know if it is exactly the same).

ref_image = np.asfarray(Image.open('avion.bmp').convert('L'))
impaired_image = np.asfarray(Image.open('avion_jpeg_r5.bmp').convert('L'))
structural_similarity(ref_image, impaired_image, multichannel=True, gaussian_weights=True, sigma=1.5, use_sample_covariance=False, data_range=255)

gives 0.8292 when I tried it.

Alternatively you can use skimage.io.imread and rgb2gray to read in the data and convert it to grayscale. In that case, values will have been scaled within [0, 1.0] for you and data_range should be set to 1.0.

from skimage.io import imread
from skimage.color import rgb2gray
ref_image = imread('avion.bmp')
ref_image = rgb2gray(ref_image)
impaired_image = imread('avion_jpeg_r5.bmp')
impaired_image = rgb2gray(impaired_image)

structural_similarity(ref_image, impaired_image, multichannel=True, gaussian_weights=True, sigma=1.5, use_sample_covariance=False, data_range=1.0)

gives 0.8265

I think the small difference between the two cases above is probably due to rgb2gray using a different luminance conversion than PIL's convert method.