Geometrical Perspective on Learning Behavior


We construct a geometrical perspective to justify the slow learning period and fast learning period during training. We plot the error surfaces and the solution space on the input space for a single neuron with two inputs. We study various training paths on this space when we run the back-propagation (BP) learning algorithm [1]. We display the relation between the learning curve and the training path. We apply this study to correctly and efficiently operate the momentum method [2] to accelerate the training.

Extracted Key Phrases

10 Figures and Tables

Cite this paper

@article{Liou2005GeometricalPO, title={Geometrical Perspective on Learning Behavior}, author={Cheng-Yuan Liou and Jau-Chi Huang and Yen-Ting Kuo}, journal={J. Inf. Sci. Eng.}, year={2005}, volume={21}, pages={721-732} }