5

3

I am using a LSTM RNN in Python and have successfully completed the prediction phase. My ultimate goal is anomaly detection. I'm hoping to have something like what you could see on Facebook Prophet, with anomalies marked as black dots below:

I've read loads of articles about how to classify with text/sequence data but there's not much on univariate time series data- only timestamps and randomly generated values with a few anomalies. One answer linked to the following method:

Anomaly detection is done by using the prediction errors as anomaly indicators.Prediction error is the difference between prediction made at time t−1 and the in-put value received at time t. The prediction errors from training data are modeled using a Gaussian distribution. The parameters of the Gaussian, mean and variance, are computed using maximum likelihood estimation (MLE). On new data, the log probability densities (PDs) of errors are calculated and used as anomaly scores: with lower values indicating a greater likelihood of the observation being an anomaly. A validation set containing both normal data and anomalies is used to set a threshold on log PD values that can separate anomalies from normal observations and incur as few false positives as possible. A separate test set is used to evaluate the model. Source

I understand overall why this method would be used but I have little idea of how to implement it with a Sequential Keras model. Does anyone have or know of some example code? Bonus if there is a way to visualize this because I don't know how I would go about that side of things either.

Please help and thank you!

This guy developed exactly the same LSTM model with unsupervised anomaly data based on errors: link. Uses Keras and Tensorflow

– Sultan1991 – 2018-11-14T06:55:29.707Actually black dots in Prophet are the actual observations not anomalies. – Emanuele Fumagalli – 2020-03-26T17:47:12.397