Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

MATLAB: Interpolation over NaNs in a time series

Question: How to locally interpolate over small lengths of NaNs?

I have a time series ("x" data sampled evenly at "t" times) that has blocks of NaNs. For example:

x = [ 1   2   4    2 3 15 10 NaN NaN NaN NaN 2 4 NaN 19 25]
t = [0.1 0.2 0.3 ...etc..]

I want to perform interpolation over the NaN.

The most elementary approach would be to just linearly interpolate from the left-most data point to the right-most data point. Eg. a line from x = 10 to x = 2 and the 4 NaN values will be assigned values from the line.

The length of the time series is ~1.5 million with ~10000 NaNs, so I don't want to incorporate data (in the interpolation) that is far away from the NaN locations. Some of the NaNs span a length of 1000-2000.

X(isnan(X)) = interp1(find(~isnan(X)), X(~isnan(X)), find(isnan(X)), 'linear'); 

will linearly interpolate over the NaN using the whole time series.

How would I interpolate locally? Linear should be sufficient. Perhaps linear interpolation incorporating a few points to the left and to the right of the NaN blocks (maybe 100-200 points). A natural neighbour or spline (?) algorithm might be more suitable; I must be careful in not adding anomalous behaviour to the time series (e.g. interpolation that adds fictitious "power" to a frequency).

UPDATE: The time series is a record of a minute-sampled temperature over a year long period. Linear interpolation is sufficient; I just need to fill in the ~6-7 hour length gaps of NaNs (I am provided with data before the NaN gaps and after the NaN gaps).

like image 398
Justin Avatar asked Nov 19 '12 01:11

Justin


People also ask

How does MATLAB deal with NaN?

Many MATLAB functions enable you to ignore missing values, without having to explicitly locate, fill, or remove them first. For example, if you compute the sum of a vector containing NaN values, the result is NaN . However, you can directly ignore NaN s in the sum by using the 'omitnan' option with the sum function.

How do you compare NaN values in MATLAB?

Description. TF = isnan( A ) returns a logical array containing 1 ( true ) where the elements of A are NaN , and 0 ( false ) where they are not. If A contains complex numbers, isnan(A) contains 1 for elements with either real or imaginary part is NaN , and 0 for elements where both real and imaginary parts are not NaN ...

How can I interpolate in MATLAB?

vq = interp1( x , v , xq ) returns interpolated values of a 1-D function at specific query points using linear interpolation. Vector x contains the sample points, and v contains the corresponding values, v(x). Vector xq contains the coordinates of the query points.

Can you plot NaN MATLAB?

If you have NaN ("Not A Number") in your data, MATLAB will ignore them in a plot. If this is what you want, that is great. If it is not, you will need to remove them.


2 Answers

I think this is (at least partially) what you seek:

% example data
x = [ 1   2   4    2 3 15 10 NaN NaN NaN NaN 2 4 NaN 19 25];
t = linspace(0.1, 10, numel(x));

% indices to NaN values in x 
% (assumes there are no NaNs in t)
nans = isnan(x);

% replace all NaNs in x with linearly interpolated values
x(nans) = interp1(t(~nans), x(~nans), t(nans));

note that you can easily switch interpolation method here:

% cubic splines
x(nans) = interp1(t(~nans), x(~nans), t(nans), 'spline');

% nearest neighbor
x(nans) = interp1(t(~nans), x(~nans), t(nans), 'nearest');
like image 129
Rody Oldenhuis Avatar answered Nov 05 '22 17:11

Rody Oldenhuis


Consider using inpaint_nans, a very nice tool designed to interpolate NaN elements in a 1-d or 2-d array using non-NaN elements. It can also extrapolate, as it does not use a triangulation of the data. It also allows different approaches to the interpolation.

like image 27
bla Avatar answered Nov 05 '22 15:11

bla