2

I am about to evaluate a neural network and want to check whether the predictions make sense.

The variables:

```
X_train1.shape
> (1700, 2)
X_val1.shape
> (339, 2)
X_test1.shape
> (701, 2)
```

hold in `[:, 0]`

the x-values (in this case time formats respectively hour-values) and in `[:, 1]`

the corresponding y-values (in this case current values):

```
X_train1[0:3]
> array([[-1.72694903, 0.63332341],
[-1.71673039, 1.00769389],
[-1.70651176, 1.14177968]])
```

You can't see time formats in the first column because I had to convert the time formats to numeric values according to

```
df_train1["timestamp"] = pd.to_timedelta(df_train1['timestamp']).dt.total_seconds()
```

Beware that I want to make use of an AutoEncoder, where `X=Y`

:

```
y_train1 = X_train1
```

and visually:

First, I search the best hyperparameters:

```
grid_result = grid_obj.fit(X_train1, y_train1)
```

Then I use the best parameters:

```
grid_best = grid_result.best_estimator_
```

Now I want to predict:

```
prediction = grid_best.predict(X_test1)
```

in principle this works but I get way too good results though I can see deviations. But I wonder if my concept is meaningful. For example, all the time before, I used `predict()`

to predict new `y`

on `X`

.
When I try to do the same above, means:

```
grid_best.predict(X_test1[:, 0])
```

I receive:

Error when checking input: expected input_152 to have shape (2,) but got array with shape (1,)

I train on a period of time of 3 months and predict on 1 month. Why I think that's too good is because I only use five neurons here: (Input 2, latent space 1, output 2). When I use six neurons, it is almost perfect..

1how are you measuring error? The time series looks fearly easy to predict. It shouldn't be surprising that a regressor fits well. Try to compare the error you get with a baseline metric, I'd suggets

`MASE`

error adjusting the seasonal parameter to your problem. – lsmor – 2019-11-05T11:57:46.120Where exactly do I define how to measure the error (for prediction)? Either it is negative mean squared error or explained variance, I'm just not sure which exact argument that is. In principle you're right. The current follows a characteristic pattern, I'm just astouned that so few neurons are so powerful and that they are able even to reconstruct few very characteristic structures. Thanks for the advice! – Ben – 2019-11-05T13:12:44.387

Can you give me another advice how to proceed? I want to detect anomalies and I wonder whether it is better to use the reconstruction error or if I should try to "predict" values (then I think I have to add a RNN to it, or?) or maybe use the AE as a classifier? – Ben – 2019-11-05T13:13:51.623

2For anomaly detection I've used statistical model such as

`Arima`

or`STL decomposition`

which give a confidence interval. I'd go that way, but you can use also ML methods. There is some literature about this. On the topic of the error, be careful. Time series are tricky, and good predictors might be bad ones when measure with the right error metric. In you case (hourly data) you can use the following predictorI'm gonna predict the next values as the same as yesterday at the same hour. For this Time series looks like a good predictor. Even better than the NN – lsmor – 2019-11-05T13:35:17.9201Thanks for the useful input! I thought about combining Arima with DL as I found that it is used sometimes. In my case I have about between 10 - 30 variables which I'll consider. I just focus on one atm and I'll add the other ones piece by piece. Predicting only in one day steps might be difficult due to trends and so on, or not? – Ben – 2019-11-05T14:30:35.283

The granularity of the predictions depends a lot on the problem. I can't give any advices. Probably, training a few models on different timesteps is an option. If your are going to use mixed models as Arima and DL try also conbining with xgboost. It is surprisingly good in time series. – lsmor – 2019-11-05T14:35:51.170

1

Thank you! Do you mean the framework/library: https://en.wikipedia.org/wiki/XGBoost ?

– Ben – 2019-11-06T06:00:58.010Yes! well... XGboost is kind of a random forest improved. Python's Scikit-learn has an implementation of the algorithim ;). Also R has a library called

`xgboost`

– lsmor – 2019-11-06T07:38:35.170