Check similarity between time series

6

2

I have time series of parameters A, B, C and D. All of them are under influence of the same major conditions, but each one has minor differences. They are placed in different locations, A, B, C are in local1 and D is in local2.

I would like to know which one (A, B, C) has the major similarity to D. How should I approach this issue?

KcFnMi

Posted 2016-12-19T11:22:03.587

Reputation: 273

What do you mean by "similarity" here? Can you define it mathematically? – Spacedman – 2016-12-20T08:18:16.780

Well, I don't have a math definition in hand right now. May I ask I guidance on this? – KcFnMi – 2017-01-02T19:37:00.240

Answers

6

Sounds like a job for Dynamic Time Warping, there are implementations in Python and R.

pgalilea

Posted 2016-12-19T11:22:03.587

Reputation: 454

4

There are many ways in which you can compute a distance between time series, and the method to use will depend on your data.

As stated by other answers, Dynamic Time Warping may be the way to go. However, this method assumes that there may be a non-linear warp between different parts of the time series.

If you are not expecting warping or delays in the signal, something as simple as Euclidean distance may be a better way to go. Of course, you should apply Euclidean distance only after you applied some preprocessing (for instance, amplitude scaling).

Take a look to this presentation, that introduces the pros and cons of these methods and discuss preprocessing in more detail:

Pablo Suau

Posted 2016-12-19T11:22:03.587

Reputation: 1 507

May I understand "amplitude scaling" as normalization? – KcFnMi – 2017-01-02T19:41:48.377

Since, "this method assumes that there may be a non-linear warp between different parts of the time series". What happens in case that thing (non-linear warp between different parts of the time series) didn't exist in the time series I'm dealing with? Should DTW still work? – KcFnMi – 2017-01-02T19:58:40.650

1Yes, you may undestand amplitude scaling as normalization. A way of doing this is by substracting the average value to each time series and then dividing each one by the width of the range of amplitude values

If non-linear warp does not exist DTW may still work, but that would depend on your data. What I would suggest is for you to try both methods (DTW and euclidean), and visualize the results so that you can determine which one works better in your specific case. – Pablo Suau – 2017-01-03T10:25:25.167

1Upvote for the awesome presentation you shared :) thanks. – mannuscript – 2018-07-14T05:00:59.987

3

First, you should define what you mean with similarity and a corresponding metric to measure it. The second step is to apply the metric to (A, D), (B, D), etc. Then, the set that gives the smallest value is the one with the highest similarity. If you find that your metric does not what you want, simply change it until it meets your requirements. You need to be clear about what you mean with "similarity" and how to measure it though.

One possibility is to interpret the sets of parameters as a point in a N-dimensional space and calculate the average euclidean distance.

Sören

Posted 2016-12-19T11:22:03.587

Reputation: 584

0

You could just check the RMSE between A, B,C to D and take the minimal one.

Binyamin Even

Posted 2016-12-19T11:22:03.587

Reputation: 225

0

In addition to the provided methods, I have found Pearson Correlation Coefficient (also mentioned in this answer) and Cosine Similarity metrics to be useful in this scenario.

In practice, Pearson Correlation Coefficient seems to be better than Cosine Similarity though. The reason is that Pearson Correlation Coefficient is invariant to scaling of a series (~adding a constant).

Here is a little proof:enter image description here

If you take a look at the output [17], cosine similarity decreases when we add a constant to the original series! On the other hand in output [18], the Pearson correlation coefficient values do not change to scaling. For this reason, in my case, I have decided to go with the Pearson Correlation Coefficient.

Bilguun

Posted 2016-12-19T11:22:03.587

Reputation: 1

1Following @Pablo Suau's answer, to note that both Pearson correlation and cosine similarity implicitly assume that there's no lag etc. between the time series. – Itamar Mushkin – 2020-08-19T07:43:13.620

@ItamarMushkin yes indeed! As I understand, Dynamic Time Warping (DTW) comes in handy in this case? – Bilguun – 2020-08-20T03:12:57.867

-1

I wrote this tutorial a while back to precisely provide guidance on these issues. It covers four ways to quantify similarity (synchrony) between time series data using Pearson correlation, time-lagged cross correlation, dynamic time warping (as mentioned earlier), and instantaneous phase synchrony. What you choose to use will depend on how you define similarity and the characteristics of your data.

Jin Cheong

Posted 2016-12-19T11:22:03.587

Reputation: 1