On derivation of MLP backpropagation from the Kelley-Bryson optimal-control gradient formula and its application

@article{Mizutani2000OnDO,
  title={On derivation of MLP backpropagation from the Kelley-Bryson optimal-control gradient formula and its application},
  author={E. Mizutani and S. Dreyfus and K. Nishio},
  journal={Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium},
  year={2000},
  volume={2},
  pages={167-172 vol.2}
}
  • E. Mizutani, S. Dreyfus, K. Nishio
  • Published 2000
  • Computer Science
  • Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium
The well-known backpropagation (BP) derivative computation process for multilayer perceptrons (MLP) learning can be viewed as a simplified version of the Kelley-Bryson gradient formula in the classical discrete-time optimal control theory. We detail the derivation in the spirit of dynamic programming, showing how they can serve to implement more elaborate learning whereby teacher signals can be presented to any nodes at any hidden layers, as well as at the terminal output layer. We illustrate… Expand
33 Citations
Online neuro-fuzzy CANFIS hidden-node teaching
  • E. Mizutani, J. Fan
  • Mathematics, Computer Science
  • 2011 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE 2011)
  • 2011
  • 2
On complexity analysis of supervised MLP-learning for algorithmic comparisons
  • E. Mizutani, S. Dreyfus
  • Computer Science
  • IJCNN'01. International Joint Conference on Neural Networks. Proceedings (Cat. No.01CH37222)
  • 2001
  • 37
A Note on Liao’s Recurrent Neural-Network Learning for Discrete Multi-stage Optimal Control Problems
  • E. Mizutani
  • Computer Science
  • Neural Processing Letters
  • 2019
  • 1
On derivation of stagewise second-order backpropagation by invariant imbedding for multi-stage neural-network learning
  • E. Mizutani, S. Dreyfus
  • Mathematics, Computer Science
  • The 2006 IEEE International Joint Conference on Neural Network Proceedings
  • 2006
  • 8
  • PDF
On practical use of stagewise second-order backpropagation for multi-stage neural-network learning
  • E. Mizutani, S. Dreyfus
  • Mathematics, Computer Science
  • 2007 International Joint Conference on Neural Networks
  • 2007
  • 2
  • PDF
MLP's hidden-node saturations and insensitivity to initial weights in two classification benchmark problems: parity and two-spirals
  • E. Mizutani, S. E. Dreyfus
  • Mathematics
  • Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290)
  • 2002
  • 12
On exploiting symmetry for multilayer perceptron learning
  • E. Mizutani, J. Fan
  • Mathematics, Computer Science
  • 2007 International Joint Conference on Neural Networks
  • 2007
  • 3
On the Use of Dropouts in Neural Networks for System Identification and Control
  • S. M. Yadav, K. George
  • Computer Science
  • 2018 IEEE Symposium Series on Computational Intelligence (SSCI)
  • 2018
Second-order backpropagation algorithms for a stagewise-partitioned separable Hessian matrix
  • 13
  • PDF
...
1
2
3
4
...

References

SHOWING 1-10 OF 27 REFERENCES
Coactive neuro-fuzzy modelling for colour recipe prediction
  • 9
Powell's dogleg trust-region steps with the quasi-Newton augmented Hessian for neural nonlinear least-squares learning
  • E. Mizutani
  • Mathematics, Computer Science
  • IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339)
  • 1999
  • 21
Gradient Theory of Optimal Flight Paths
  • 293
  • PDF
Learning internal representations by error propagation
  • 18,432
  • PDF
Color device characterization of electronic cameras by solving adaptive networks nonlinear least squares problems
  • 6
Chapter 6: Derivative-based Optimization. In Neuro-Fuzzy and Soft Com- puting: a computational approach to learning and machine intelligence, pages 129{172
  • 1997
Chapter 6: Derivative-based Optimization. In Neuro-Fuzzy and Soft Computing: Q computational approach t o learning and machine intelligence, pages 129-172
  • 1997
Chapter 6: Derivative-based Optimization. In Neuro-Fuzzy and Soft Computing: a computational approach to learning and machine intelligence
  • Chapter 6: Derivative-based Optimization. In Neuro-Fuzzy and Soft Computing: a computational approach to learning and machine intelligence
  • 1997
Chapter 6: Derivative-based Optimization. In Neuro-Fuzzy and Soft Computing: a computational approach to learning and machine intelligence, pages 129–172
  • 1997
...
1
2
3
...