How to find patterns in a series of timestamps

4

I have a series of timestamps that represent the time a user clicked a certain button.

My goal is to detect the automated clicks, so I need to find recurring patterns in the data that may point to an automated script. The majority of the data is by regular users. I don't need to detect them online, I just need to detect them in historical data. I also have user_id to make a distinction between users.

Is there a default approach to detecting recurring patterns in a largely unstructured dataset?

Most things I've found are about anomaly detection on a certain value paired with a timestamp but my problem is not really anomaly-detection as I need to find patterns amongst unpatterned data

Deb

Posted 2017-08-21T08:43:43.737

Reputation: 141

Welcome to the site! Do you need to detect it as it happens? Are there multiple buttons you need to track? This is an anomaly detection problem. – Emre – 2017-08-21T08:47:25.893

I don't need to detect it as it happens, I just need to detect it in historical data. It's one event/button I need to track. – Deb – 2017-08-21T08:48:39.447

Do you also have some training data that distinguishes bots from users? – Emre – 2017-08-21T08:56:05.897

@Emre Unfortunately not – Deb – 2017-08-21T08:59:13.573

Answers

2

The question can be super difficult if you only have the number of clicks per time-stamp. The reason is that you might find many different recurrent patterns! For instance certain time periods in year, month or day (according to the functionality of website) may attract a certain temporal pattern of clicks. But if you have more relevant features the story is different.

After all I will point out some directions and hope it helps:

Autocorrelation may initially give you a good insight about what is happening there.

Then go for Time-Warping methods to define similarity function and seek it along time-series.

Embedding in high-dimensional spaces disclose the temporal recurrent pattern of a time-series. It's mostly from physics literature and is called State-Space Recunstruction. This method transfers your time-series into a m-dimensional space using this and there you can see the time-series as a dynamic system which may help in detecting recurrent dynamics.

Kasra Manshaei

Posted 2017-08-21T08:43:43.737

Reputation: 5 323

Thank you for the info, I will checkout the links! Other relevant data I have are user id's so I am sure to only find patterns from a single user/bot. – Deb – 2017-08-21T09:27:12.443