Anomaly detection for transaction data

4

3

I have transaction details for credit data (bank transfers, peer to peer transfers, etc). Currently, I have one year worth of data which I cannot properly classify.

I'm looking for input and suggestions about these two questions:

  1. How do I detect anomalies in the last hour with previous credit transaction data that I have?

  2. How can I detect anomalies in real-time for the current transaction?

Kira

Posted 2017-02-13T12:17:42.503

Reputation: 41

1Hi. Welcome to the site :) – Dawny33 – 2017-02-13T12:22:16.143

Answers

3

There are a few factors to consider in anomaly detection. A simple method would be to plot a boxplot of the data and calculate outliers this way (boxplot description).

If the time series is seasonal, you could take an approach similar to Twitter's anomaly detection algorithm (Twitter Anomaly Detection). This uses an outlier detection method (ESD) applied to the seasonal decomposition of the time series.

You could use clustering and SVMs as well.

There is also change-point detection which I understand less but can be very effective.

Really, your answer depends on how complex your data is. Sometimes, a simple heuristic algorithm works the best.

Here are some similar questions:

Anomaly detection in Time Series Data - Help Required

How to classify and cluster this time series data

Hobbes

Posted 2017-02-13T12:17:42.503

Reputation: 1 329

There is no seasonal data. Transactions are uniform. There is no training data. Would time-series work fine? – Kira – 2017-02-14T07:21:43.560

I would avoid any time series approach that specifically addresses seasonality. Start simple and work your way up. Some quick boxplots to visualize your data, and see if this captures your outliers. ESD (http://www.itl.nist.gov/div898/handbook/eda/section3/eda35h3.htm) could be a useful method. Clustering could also work, and it is unsupervised, however, there are some pitfalls to k-means. Without knowing the data, it is hard to say what time series approaches will be appropriate but it's definitely worth a look.

– Hobbes – 2017-02-14T16:03:00.443