Plotting shaded uncertainty region in line plot in matplotlib when data has NaNs

Question

I would like a plot which looks like this:

I am trying to do this with matplotlib:

fig, ax = plt.subplots()  with sns.axes_style("darkgrid"):     for i in range(5):         ax.plot(means.ix[i][list(range(3,104))], label=means.ix[i]["label"])         ax.fill_between(means.ix[i][list(range(3,104))]-stds.ix[i][list(range(3,104))], means.ix[i][list(range(3,104))]+stds.ix[i][list(range(3,104))])     ax.legend() 

I want the shaded region to be the same colour as the line in the centre. But right now, my problem is that means has some NaNs and fill_between does not accept that. I get the error

TypeError: ufunc 'isfinite' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''

Any ideas on how I could achieve what I want? The solution doesn't need to use matplotlib as long as it can plot my series of points with their uncertainties for multiple series.

Answer 1

Ok. So one of the problem was that the dtype of my data was object and not float and this caused fill_between to fail when it looked to see if the numbers were finite. I finally managed to do it by (a) converting to float and then (b) to solve the problem of the matching colours for uncertainty and line, to use a colour palette. So I have:

import seaborn as sns import numpy as np import matplotlib.pyplot as plt import pandas as pd fig, ax = plt.subplots() clrs = sns.color_palette("husl", 5) with sns.axes_style("darkgrid"):     epochs = list(range(101))     for i in range(5):         meanst = np.array(means.ix[i].values[3:-1], dtype=np.float64)         sdt = np.array(stds.ix[i].values[3:-1], dtype=np.float64)         ax.plot(epochs, meanst, label=means.ix[i]["label"], c=clrs[i])         ax.fill_between(epochs, meanst-sdt, meanst+sdt ,alpha=0.3, facecolor=clrs[i])     ax.legend()     ax.set_yscale('log') 

which gave me the following result:

Answer 2

You could simply drop the NaNs from your means DataFrame and plot that resulting dataframe instead?

In the example below, I tried to get close to your structure, I have a means DataFrame with some NaN sprinkled around. I suppose the stds DataFrame probably has NaN at the same locations, but in this case it doesn't really matter, I drop the NaN from means to get temp_means and I use the indices left in temp_means to extract the std values from stds.

The plots show the results before (top) and after (bottom) dropping the NaNs

x = np.linspace(0, 30, 100) y = np.sin(x/6*np.pi) error = 0.2  means = pd.DataFrame(np.array([x,y]).T,columns=['time','mean']) stds = pd.DataFrame(np.zeros(y.shape)+error)  #sprinkle some NaN in the mean sprinkles = means.sample(10).index means.loc[sprinkles] = np.NaN   fig, axs = plt.subplots(2,1)  axs[0].plot(means.ix[:,0], means.ix[:,1]) axs[0].fill_between(means.ix[:,0], means.ix[:,1]-stds.ix[:,0], means.ix[:,1]+stds.ix[:,0])  temp_means = means.dropna()  axs[1].plot(temp_means.ix[:,0], temp_means.ix[:,1]) axs[1].fill_between(temp_means.ix[:,0], temp_means.ix[:,1]-stds.loc[temp_means.index,0], temp_means.ix[:,1]+stds.loc[temp_means.index,0])   plt.show()