specific outliers on a heat map- matplotlib

Question

I am generating a heat map with data that has a fixed outlier number and I need to show these outliers as a colour out of the colour palette of the cmap I use which is "hot". With the use of cmap.set_bad('green') and np.ma.masked_values(data, outlier), I get a plot which looks right but the color bar is not getting synced with the data properly even if I use cmap.set_over('green'). Here is the code I have been trying:

plt.xlim(0,35)
plt.ylim(0,35)
img=plt.imshow(data, interpolation='none',norm=norm, cmap=cmap,vmax=outlier)

cb_ax=fig.add_axes([0.85, 0.1, 0.03, 0.8])

cb=mpl.colorbar.ColorbarBase(cb_ax,cmap=cmap,norm=norm,extend='both',spacing='uniform')
cmap.set_over('green')
cmap.set_under('green')

Here is the data (outlier is 1.69 obviously):

Data;A;B;C;D;E;F;G;H;I;J;K    
A;1.2;0;0;0;0;1.69;0;0;1.69;1.69;0    
B;0;0;0;0;0;1.69;0;0;1.69;1.69;0    
C;0;0;0;0;0;1.69;0;0.45;1.69;1.69;0.92    
D;1;0;-0.7;-1.2;0;1.69;0;0;1.69;1.69;0    
E;0;0;0;0;0;1.69;0;0;1.69;1.69;0    
F;1.69;1.69;1.69;1.69;1.69;1.69;1.69;1.69;1.69;1.69;1.69    
G;0;0;0;0;0;1.69;0;0;1.69;1.69;0    
H;0;0;0;0;0;1.69;0;0;1.69;1.69;0    
I;1.69;1.69;1.69;1.69;1.69;1.69;1.69;1.69;1.69;1.69;1.69
J;1.69;1.69;1.69;1.69;1.69;1.69;1.69;1.69;1.69;1.69;1.69
K;0;0;0;0;0;1.69;0;0;1.69;1.69;0

Appreciate any help

Answer 1

What's happening is that you're using a masked array where the outliers are masked.

Therefore, they don't show up on the colorbar as being "over". (i.e. as far as matplotlib is concerned, the masked values are invalid, not over the threshold)

As a stand-alone example to reproduce your problem:

import numpy as np
import matplotlib.pyplot as plt

threshold = 0.8
data = np.random.random((10,10))
data = np.ma.masked_greater(data, threshold)

fig, ax = plt.subplots()
im = ax.imshow(data, cmap=plt.cm.hot, interpolation='none')
cbar = fig.colorbar(im, extend='max')
cbar.cmap.set_over('green')

plt.show()

If we simply don't make this a masked array, and instead specify the vmax kwarg to imshow:

import numpy as np
import matplotlib.pyplot as plt

threshold = 0.8
data = np.random.random((10,10))

fig, ax = plt.subplots()
im = ax.imshow(data, cmap=plt.cm.hot, interpolation='none', vmax=threshold)
cbar = fig.colorbar(im, extend='max')
cbar.cmap.set_over('green')

plt.show()

Basically, this is the difference between set_over (or under) and set_bad.

If you did still want to use a masked array, you could just call cbar.cmap.set_bad('green') as well as set_over, and you'd get the effect you want (though all "bad" values, not just ones over the threshold, would be green). If you take that route, you'll need to manually specify the vmax. Otherwise it will be taken as the maximum of the unmasked portions of the array.

Answer 2

I think you need to set extend to "both" and feed in a Normalize object:

import numpy as np
import matplotlib as mpl
import matplotlib.pyplot as plt
import pandas

from io import StringIO # python 3
#from StringIO import StringIO # python 2

datastring = StringIO("""\
Data;A;B;C;D;E;F;G;H;I;J;K
A;1.2;0;0;0;0;1.69;0;0;1.69;1.69;0
B;0;0;0;0;0;1.69;0;0;1.69;1.69;0
C;0;0;0;0;0;1.69;0;0.45;1.69;1.69;0.92
D;1;0;-0.7;-1.2;0;1.69;0;0;1.69;1.69;0
E;0;0;0;0;0;1.69;0;0;1.69;1.69;0
F;1.69;1.69;1.69;1.69;1.69;1.69;1.69;1.69;1.69;1.69;1.69
G;0;0;0;0;0;1.69;0;0;1.69;1.69;0
H;0;0;0;0;0;1.69;0;0;1.69;1.69;0
I;1.69;1.69;1.69;1.69;1.69;1.69;1.69;1.69;1.69;1.69;1.69
J;1.69;1.69;1.69;1.69;1.69;1.69;1.69;1.69;1.69;1.69;1.69
K;0;0;0;0;0;1.69;0;0;1.69;1.69;0
""")

threshold = 1.68
data = pandas.read_table(datastring, sep=';', index_col='Data')
cmap = mpl.cm.coolwarm
norm = mpl.colors.Normalize(vmin=-1 * threshold, vmax=threshold)
cmap.set_over('slategray')
cmap.set_under('forestgreen')

fig, ax = plt.subplots()
ax.set_aspect('equal')
cb_ax=fig.add_axes([0.85, 0.1, 0.03, 0.8])
img = ax.imshow(data, cmap=cmap, norm=norm, interpolation='none')
cb = mpl.colorbar.ColorbarBase(cb_ax, cmap=cmap, norm=norm, extend='both')

Gives me: