Evaluation of Robustness and Performance of Early Stopping Rules with Multi Layer Perceptrons

Aleksander Lodwich, Yves Rangoni, Thomas Breuel
IJCNN 2009, Atlanta, Georgia, USA, IEEE, IEEE and INNS, 6/2009

Abstract:

In this paper, we evaluate different Early Stopping Rules (ESR) and their combinations for stopping the training of Multi Layer Perceptrons (MLP) with stochastic gradient descent, also known as online error backpropagation, before reaching a predefined maximum number of epochs. We restricted our evaluation to classification tasks as we want to compute the accuracy. Early stopping is important for two reasons. On one hand it prevents overfitting and on the other hand it can dramatically reduce the training time. Today, there exists an increasing amount of applications involving unsupervised and automatic training like in i.e. ensembles, where automatic stopping rules are necessary for keeping training time low. Current literature is not so specific about endorsing which rule to use, when to use it or what its robustness is. Therefore this issue is revisited in this paper. We tested on PROBEN1 and MNIST.

BibTex:

@inproceedings{ LODW2009,
	Title = {Evaluation of Robustness and Performance of Early Stopping Rules with Multi Layer Perceptrons},
	Author = {Aleksander Lodwich and Yves Rangoni and Thomas Breuel},
	BookTitle = {IJCNN 2009},
	Month = {6},
	Year = {2009},
	Publisher = {IEEE},
	Organization = {IEEE and INNS}
}

     
Last modified:: 30.08.2016