How to find tops and bottoms in time series?

Question

At first, this question can sound really stupid, but it is not in fundamental. Maybe, it can seem like unresolvable exactly by any algorithm, but I pretend to say it is.

So question. I have chart, for example gold. I need to find where are tops and bottoms on time axial. The problem is I need to find where major upturns and major downturns start. The problem is that there is lot of small irrelevant upturns and downturns.

Here is the picture for better understanding - the red spots are that I want to find(NOT EXACTLY, but in some way like this).

So I probably need to filter out small turnups and turndowns, but have no idea how to do it. I will be pleased by any ideas. I do not need algorithm in java etc, just in words it would be enough.

Answer 1

1. You could perform a smoothing or lowpass filtering operation first, and find the locations of the local minima/maxima from the smoothed data. Then get the values of the minima and the maxima from the original data.

2. You could use a normal maximum/minimum filter, which finds all turning points, then filter the list of turning points by threshold.

3. I think what you really want to do is remove the "long-term variation" from the signal and look only at the "short term variation". This is can be done using the empirical mode decomposition. See Sec 2.3.2 of my thesis. (Alernately, Google around for "Empirical Mode Decomposition", "EMD", or "Hilbert-Huang Transform".)

Here's the EMD in action:

Notice the increasing generality as the EMD algorithm extracts components of the signal, starting at "most detailed" and ending with "most general trend". (Note there are apparently nine components - only a few are shown.)