2-d interpolation ignoring nan values

Question

How do you tell interp2d to ignore nan values?

I have a surface x and y with some arbitrary value z.

x =   np.array([[9.19632, 9.62141, 10.0829, np.isnan, np.isnan],
        [9.21164, 9.64347, 10.1392, 10.5698,  np.isnan],
        [9.22175, 9.65439, 10.1423, 10.6301, 11.0323],
        [9.21632, 9.67060, 10.1474, 10.6230, 11.0818]])

y =  np.array([[11.5466,11.6485,11.7619, np.isnan, np.isnan],
        [12.4771, 12.5460, 12.5453, 12.7142, np.isnan],
        [13.5578, 13.5581, 13.5505, 13.5309, 13.6081],
        [14.5653, 14.5504, 14.5036, 14.5145, 14.5060]])

z = np.array([[0.466113, 0.0484404, -0.385355, np.isnan, np.isnan],
        [0.366125, -0.160165, -0.548668, -0.888301,np.isnan],
        [-0.0970777, -0.346734, -0.826576, -1.08412, -1.33129],
        [-0.259981, -0.586938, -1.03477, -1.32384, -1.61500]])

I have been able to produce the above colour mesh using a masked array but I'm failing when I try an create a finer grid using 2d interpolation. Below is what I have so far, note that I set the nan values to zero to obtain this so obviously it messes up the 'correct' interpolation. Instead I would like to ignore them and leave the parameter space blank.

f = interp.interp2d(x,y,z, kind='linear')
xnew = np.arange(9,11.5, 0.01)
ynew = np.arange(9,15, 0.01)
znew = f(xnew, ynew)


levels = np.linspace(zmin, zmax, 15)
plt.ylabel('Y', size=15)
plt.xlabel('X', size=15)
cmap = plt.cm.jet_r
cmap.set_bad('white',0.1) # set nan to white
cs = plt.contourf(xnew, ynew, znew, levels=levels, cmap=cmap)
cbar = plt.colorbar(cs)
cbar.set_label('Z', rotation=90, fontsize=15) # gas fraction
plt.show()

I want to simply create a smooth colour plot where the regions bound by x and y are coloured according to z.

Answer 1

I don't know why interp2d has problems with irregular spaced data, I would recommend using griddata, you can flat your input data into a vector with ravel and then eliminate the NaN, and use it as input of griddata, you would get something like this

Code is not much different from what you have

import numpy as np
import matplotlib.pyplot as plt
from scipy.interpolate import griddata

x =   np.array([[9.19632, 9.62141, 10.0829,np.isnan,np.isnan],
    [9.21164, 9.64347, 10.1392, 10.5698,np.isnan],
    [9.22175, 9.65439, 10.1423, 10.6301, 11.0323],
    [9.21632, 9.67060, 10.1474, 10.6230, 11.0818]])

y =  np.array([[11.5466,11.6485,11.7619,np.isnan,np.isnan],
    [12.4771, 12.5460, 12.5453, 12.7142,np.isnan],
    [13.5578, 13.5581, 13.5505, 13.5309, 13.6081],
    [14.5653, 14.5504, 14.5036, 14.5145, 14.5060]])

z = np.array([[0.466113, 0.0484404, -0.385355,np.isnan,np.isnan],
    [0.366125, -0.160165, -0.548668, -0.888301,np.isnan],
    [-0.0970777, -0.346734, -0.826576, -1.08412, -1.33129],
    [-0.259981, -0.586938, -1.03477, -1.32384, -1.61500]])

x=x.ravel()              #Flat input into 1d vector
x=list(x[x!=np.isnan])   #eliminate any NaN
y=y.ravel()
y=list(y[y!=np.isnan])
z=z.ravel()
z=list(z[z!=np.isnan])


xnew = np.arange(9,11.5, 0.01)
ynew = np.arange(9,15, 0.01)
znew = griddata((x, y), z, (xnew[None,:], ynew[:,None]), method='linear')



levels = np.linspace(min(z), max(z), 15)
plt.ylabel('Y', size=15)
plt.xlabel('X', size=15)
cmap = plt.cm.jet_r
cs = plt.contourf(xnew, ynew, znew, levels=levels, cmap=cmap)
cbar = plt.colorbar(cs)
cbar.set_label('Z', rotation=90, fontsize=15) # gas fraction
plt.show()

---

If you must extrapolate data (check my comment below) you can use SmoothBivariateSpline and play around with the order of the spline, I don't recommend it, I'll show you why.

The code is closer to what you had originally.

from scipy.interpolate import SmoothBivariateSpline

x=x.ravel()
x=(x[x!=np.isnan])
y=y.ravel()
y=(y[y!=np.isnan])
z=z.ravel()
z=(z[z!=np.isnan])

xnew = np.arange(9,11.5, 0.01)
ynew = np.arange(10.5,15, 0.01)

f = SmoothBivariateSpline(x,y,z,kx=1,ky=1)

znew=np.transpose(f(xnew, ynew))

With kx=1 and ky=1 f = SmoothBivariateSpline(x,y,z,kx=1,ky=1) you get

With kx=2 and ky=2 you get:

And with kx=3 and ky=3 you get:

I changed the levels on the 3 pictures so it's easier to see. But check the scale, the values can get crazy very fast outside your sampling area, so if you MUST extrapolate, do it with diligence