Tutorial :How to detect a trend inside unsteady data (e.g. Trendly)?



Question:

I was wondering what kind of model / method / technique Trendly might use to achieve this model:

alt text

[It tries to find the moments where significant changes set in and ignores random movements]

Any pointers very welcome! :)


Solution:1

I've never seen 'Trendly', and don't know anything about it, but if I wanted to produce that red line from that blue line, in an algorithmic fashion, I would try:

  1. Fourier the whole data set
  2. Choose a block size longer than the period of the dominant frequency
  3. Divide the data up into blocks of the chosen size
  4. Compare adjacent ones with a statistical test of some sort.
  5. Where the test says two blocks belong to the same underlying distribution, merge them.
  6. If any were merged, go back to 4.
  7. Red trend line is the mean of each block.


Solution:2

A simple "median" function could produce smoother curves over a mostly un-smooth curve.

Otherwise, a brute-force or genetic algorithm could be used; attempting to find the way to split the data into sections, so that more sections = worse solution, and less accuracy of the lines = worse solution.

Another way would be like this: Start at the beginning. As soon as the line moves outside of some radius (3 above or 3 below the first, for instance) set the new height to an average of the current line's height and the previous marker.
If you keep doing that, it would ignore small fluctuations. However, if the fluctuation was large enough, it would still effect it.


Note:If u also have question or solution just comment us below or mail us on toontricks1994@gmail.com
Previous
Next Post »