Rapid change detection algorithm

Question

I'm logging temperature values in a room, saving them to the database. I'd like to be alerted when temperature rises suddenly. I can't set fixed values, because 18°C is acceptable in winter and 25°C is acceptable in summer. But if it jumps from 20°C to 25°C during, let's say, 30 minutes and stays like this for 5 minutes (to eliminate false readouts), I'd like to be informed.

My current idea is to take readouts from last 30 minutes (A) and readouts from last 5 minutes (B), calculate median of A and B and check if difference between them is less then my desired threshold.

Is this correct way to solve this or is there a better algorithm? I searched for a specific one but most of them seem overcomplicated.

Thanks!

Answer 1

Detecting changes in a time-series is a well-researched subject, and hundreds if not thousands of papers have been written on this subject. As you've seen many methods are quite advanced, but proved to be quite useful for many use cases. Whatever method you choose, you should evaluate it against real of simulated data, and optimize its parameters for your use case.

As you require, let me suggest a very simple method that in many cases prove to be good enough, and is quite similar to that you considered.

Basically, you have two concerns:

• Detecting a monotonous change in a sampled noisy signal
• Ignoring false readouts

First, note that medians are not commonly used for detecting trends. For the series (1,2,3,30,35,3,2,1) the medians of 5 consecutive terms is be (3, 3, 3, 3). It is much more common to use averages.

One common trick is to throw the extreme values before averaging (e.g. for each 7 values average only the middle 5). If many false readouts are expected - try to take measurements at a faster rate, and throw more extreme values (e.g. for each 13 values average the middle 9).

Also, you should throw away unfeasible values and replace them with the last measured value (unfeasible means out of range, or non-physical change rate).

Your idea of comparing a short-period measure with a long-period measure is a good idea, and indeed it is commonly used (e.g. in econometrics).

Quoting from "Financial Econometric Models - Some Contributions to the Field [Nicolau, 2007]:

Buy and sell signals are generated by two moving averages of the price level: a long-period average and a short-period average. A typical moving average trading rule prescribes a buy (sell) when the short-period moving average crosses the long-period moving average from below (above) (i.e. when the original time series is rising (falling) relatively fast).