Menu
As indicated in the title, I am wondering if the DTW (Dynamic Time Warping) could be used to calculate the DTW distance between two time series with missing values.
Let's say the two time series are daily temperatures of two weather stations, and are of equal lengths (e.g. 365 days), and the missing values are on different days for the two time series.
If this is possible, is the dtw package in R able to handle the missing values? I didn't find a parameter that could be set in dtw() like na.rm = T.
Thanks a lot!
Thanks thelatemail for the suggestion. Below is a simplified example of the two time series, where each time series contain only 52 elements and the missing values are set to NA.
TS1 = c(-3.26433, -5.09096, NA, -8.4158, -5.85485, -3.49234, -7.64666, -4.90124, NA, -4.68836, -1.38114, 1.55527, 2.81872, 2.44261, 3.57963, 6.19983, 7.42515, 8.41524, 6.32686, 10.0144, 9.53251, 13.4781, 12.3585, 10.6706, 10.2647, 16.6848, 16.4855, 20.1482, NA, 21.5734, 20.3946, 20.8824, 18.0325, 18.5813, 17.5453, 16.3315, 14.3068, 11.3164, 9.96398, 5.53102, 9.55094, 9.05897, 6.81199, 5.20343, 1.63158, -0.661077, -4.33853, -6.53655, NA, -10.8646, 1.11843, 1.23786)
TS2 = c(-5.76852, -10.2207, -11.8465, NA, -1.70019, -3.60319, -5.7718, -3.81106, -5.62284, -3.57516, 0.314511, 0.64058, 0.476162, NA, 4.23757, 5.15417, 7.29422, NA, 1.57376, 9.28236, 8.05182, 13.7175, 9.5453, 10.2417, 9.32423, 18.214, 18.3726, 16.661, 20.6563, 22.2901, 22.1109, 19.129, 15.8615, 16.7817, 17.247, 15.9921, 14.5804, 11.3693, 10.9349, 10.1196, 3.7467, 9.09229, 6.91285, NA, 4.20934, -0.566403, -2.94184, -3.81432, -10.0212, -15.9876, -2.56286, -1.88976)
804
Dynamic Time Warping is used to compare the similarity or calculate the distance between two arrays or time series with different length. How to do that? One obvious way is to match up a and b in 1-to-1 fashion and sum up the total distance of each component.
In general, the inter-word dissimilarity measure supplied by Dynamic Time Warping algorithms can not be assumed to be a metric because it does not fully satisfy all the required properties (the triangle inequality in particular).
Dynamic Time warping is a method of calculating distance that is more accurate than Euclidean distance. It has an advantage over Euclidean if datapoints are shifted between each other and we want to look rather at its shape.
Dynamic Time Warping (DTW) is an A.I. technique which has been very useful for normalizing and comparing data with unequal lengths of data. Similarly, there are key inputs of unequal lengths and varying time speeds.
Probably not, I looked over the package manual and there is nothing about the missing or NA values. I also tried to feed your data to dtw() and it fails:
Error in dtw(TS1, TS2) :
No warping paths exists that is allowed by costraints
But when I changed all NA values to 0, it worked easily.
So if your only solution is this package, you can make a post on the DTW package forum, or probably you have to deal the missing data yourself. You may find some hints here or use the *.na() function of the fSeries package
*This package is no longer available. It is suggested to use the timeSeries package instead.
151
I also run into this situation. The reason you are getting error message when using DTW with a time series containing NA values is that the warping distance will be undetermined when NA is present in the DTW path. I suggest you impute the NA values using some ARIMA model and then use DTW. Check out this or this for imputing missing time series values.
30
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
