Can the number of epochs influence overfitting?

15

3

I am using a convolution neural network ,CNN. At a specific epoch, I only save the best CNN model weights based on improved validation accuracy over previous epochs.

Does increasing the number of epochs also increase over-fitting for CNNs and deep learning in general?

user121

Posted 2018-02-07T13:34:05.250

Reputation: 309

Answers

14

Yes, it may. In machine-learning there is an approach called early stop. In that approach you plot the error rate on training and validation data. The horizontal axis is the number of epochs and the vertical axis is the error rate. You should stop training when the error rate of validation data is minimum. Consequently if you increase the number of epochs, you will have an over-fitted model.

In deep-learning era, it is not so much customary to have early stop. There are different reasons for that but one of them is that deep-learning approaches need so much data and plotting the mentioned graph would be so much wavy because these approach use stochastic-gradient-like optimizations. In deep-learning again you may have an over-fitted model if you train so much on the training data. To deal with this problem, another approaches are used for avoiding the problem. Adding noise to different parts of models, like drop out or somehow batch normalization with a moderated batch size, help these learning algorithms not to over-fit even after so many epochs.

In general too many epochs may cause your model to over-fit the training data. It means that your model does not learn the data, it memorizes the data. You have to find the accuracy of validation data for each epoch or maybe iteration to investigate whether it over-fits or not.

Media

Posted 2018-02-07T13:34:05.250

Reputation: 12 077

0

YES. Increasing number of epochs over-fits the CNN model. This happens because of lack of train data or model is too complex with millions of parameters. To handle this situation the options are

  1. we need to come-up with a simple model with less number of parameters to learn
  2. add more data by augmentation
  3. add noise to dense or convolution layers
  4. add drop-out layers
  5. add l1 or l2 regularizers
  6. add early stopping
  7. check the model accuracy on validation data
  8. early stopping will tell you appropriate epochs without overfitting the model

Padi Sarala

Posted 2018-02-07T13:34:05.250

Reputation: 1