On-line learning of non-monotonic rules by simple perceptron

  title={On-line learning of non-monotonic rules by simple perceptron},
  author={Jun-ichi Inoue and Hidetoshi Nishimori and Yoshiyuki Kabashima},
  journal={arXiv: Condensed Matter},
We study the generalization ability of a simple perceptron which learns unlearnable rules. The rules are presented by a teacher perceptron with a non-monotonic transfer function. The student is trained in the on-line mode. The asymptotic behaviour of the generalization error is estimated under various conditions. Several learning strategies are proposed and improved to obtain the theoretical lower bound of the generalization error. 
5 Citations
Statistical Mechanics of On-line Learning when a Moving Teacher Goes around an Unlearnable True Teacher
It is shown that the generalization errors of a student can temporarily become smaller than that of a moving teacher and can reach the lowest value, even if the student only uses examples from the moving teacher.
On-line Learning of an Unlearnable True Teacher through Mobile Ensemble Teachers
The generalization performance of the student is shown to exceed that of the ensemble teachers in a transient state, as was shown in similar ensemble-teachers models.
Statistical mechanics of ensemble learning using stochastic filtering
We analyze ensemble learning using Murata’s stochastic filtering with statistical mechanical method based on online learning theory. It is shown that the filtering is effective to keep students’


Introduction to the theory of neural computation
This book is a detailed, logically-developed treatment that covers the theory and uses of collective computational networks, including associative memory, feed forward networks, and unsupervised learning.
Neural Computation
Lecture Notes for the MSc/DTC module. The brain is a complex computing machine which has evolved to give the ttest output to a given input. Neural computation has as goal to describe the function of
J. Phys. A: Math. Gen
  • J. Phys. A: Math. Gen
  • 1996
Phys. Rev. Lett
  • Phys. Rev. Lett
  • 1996
Phys. Rev. E
  • Phys. Rev. E
  • 1995
Europhys. Lett
  • Europhys. Lett
  • 1994
J. Phys. A: Math. Gen
  • J. Phys. A: Math. Gen
  • 1993
Phys. Rev. A
  • Phys. Rev. A
  • 1992
Trans. IEICE J73-D-II
  • Trans. IEICE J73-D-II
  • 1990