Visualizing a heatmap matrix on to an image in OpenCV

Question

I have a (float32) heatmap matrix in my Python program like this:

[[0.99782705 0.99782705 0.99782705 0.99782705 0.99782705 0.99782705
  0.99782705 0.99782705 0.99782705]
 [0.99782705 0.99782705 0.99782705 0.99782705 0.99782705 0.99782705
  0.99782705 0.99782705 0.99782705]
 [0.99782705 0.99782705 0.99782705 0.99782705 0.99782705 0.99782705
  0.99782705 0.99782705 0.99782705]
 [0.99782705 0.99782705 0.99782705 0.99782705 0.99782705 0.99782705
  0.99782705 0.99782705 0.99782705]
 [0.99782705 0.99782705 0.99782705 0.99782705 0.99782705 0.99782705
  0.99919313 0.99782705 0.99782705]
 [0.99782705 0.99782705 0.99782705 0.99782705 0.99782705 0.99782705
  1.         0.99782705 0.99782705]
 [0.99782705 0.99782705 0.99782705 0.99782705 0.99782705 0.99782705
  0.99782705 0.99782705 0.99782705]
 [0.99782705 0.99782705 0.99782705 0.99782705 0.99782705 0.99782705
  0.99782705 0.99782705 0.99782705]
 [0.99782705 0.99782705 0.99782705 0.99782705 0.99782705 0.99782705
  0.99782705 0.99782705 0.99782705]]

Here is what the heatmap looks like when printed via matplotlib.pyplot.matshow():

Now I want to resize that matrix to the size of an image and overlay it as an heatmap on to that image. So first I load the other image and resize the heatmap to the image size:

img = cv2.imread(image_path)
heatmap = cv2.resize(heatmap, (img.shape[1], img.shape[0]))

After resizing the heatmap matrix becomes this:

Ok, so far everything looks well. Now, based on various resources I found online, I want to transform that heatmap into an uint8 RGB format, apply the cv2.COLORMAP_JET color map, and overlay it on to the original image:

heatmap = cv2.applyColorMap(np.uint8(255 * heatmap), cv2.COLORMAP_JET)
superimposed = heatmap * 0.4 + img

But that does not seem to work. When I render the heatmap now via cv2.imshow('Heatmap', heatmap) it becomes a plain (red) image, it has lost all its "features" from the original heatmap. Therefore also the image I want to lay the heatmap over stays the same as before (i.e. rendering superimposed looks exactly as img).

Any ideas what I might be missing here?

For ther records, I want to achieve something similar to this:
Grad-CAM implementation for Keras models

PS: Full working example here

Answer 1

If it is for visualization, you will need to make a bigger separation between the numbers. You have basically 3 numbers and when you scale them (multiply 255) they become basically 2 numbers (254 and 255). When it applies the color, both of the numbers are almost the same, and it is really hard to know which one is which...

Solution:

Create a new scale with the current numbers. You can easily do that with cv2.normalize

import matplotlib.pyplot as plt
import numpy as np
import cv2

heatmap = np.array([
    [0.99782705,0.99782705,0.99782705,0.99782705,0.99782705,0.99782705,0.99782705,0.99782705,0.99782705],
    [0.99782705,0.99782705,0.99782705,0.99782705,0.99782705,0.99782705,0.99782705,0.99782705,0.99782705],
    [0.99782705,0.99782705,0.99782705,0.99782705,0.99782705,0.99782705,0.99782705,0.99782705,0.99782705],
    [0.99782705,0.99782705,0.99782705,0.99782705,0.99782705,0.99782705,0.99782705,0.99782705,0.99782705],
    [0.99782705,0.99782705,0.99782705,0.99782705,0.99782705,0.99782705,0.99919313,0.99782705,0.99782705],
    [0.99782705,0.99782705,0.99782705,0.99782705,0.99782705,0.99782705,1.00000000,0.99782705,0.99782705],
    [0.99782705,0.99782705,0.99782705,0.99782705,0.99782705,0.99782705,0.99782705,0.99782705,0.99782705],
    [0.99782705,0.99782705,0.99782705,0.99782705,0.99782705,0.99782705,0.99782705,0.99782705,0.99782705],
    [0.99782705,0.99782705,0.99782705,0.99782705,0.99782705,0.99782705,0.99782705,0.99782705,0.99782705]
])

heatmap = cv2.resize(heatmap, (400,300))
plt.matshow(heatmap)
plt.show()
heatmapshow = None
heatmapshow = cv2.normalize(heatmap, heatmapshow, alpha=0, beta=255, norm_type=cv2.NORM_MINMAX, dtype=cv2.CV_8U)
heatmapshow = cv2.applyColorMap(heatmapshow, cv2.COLORMAP_JET)
cv2.imshow("Heatmap", heatmapshow)
cv2.waitKey(0)

and you get:

Note: I just put numbers in the resize, since I do not have the sample image.

However, there is one thing to consider here. With this solution the biggest number will be red, independently of how small it is. If you need a new fixed scaling (e.g. 1. is red and 0 is the samllest number) you need to do it manually with something like:

newvalue= (maxNew-minNew)/(max-min)*(value-max)+maxNew

where maxNew= 255 and minNew=0 and max and min will be something you arbitrarily decide (e.g. 0.9978 and 1.0)