Corpus ID: 220055484

An Effective and Efficient Initialization Scheme for Training Multi-layer Feedforward Neural Networks

@inproceedings{Yang2020AnEA,
  title={An Effective and Efficient Initialization Scheme for Training Multi-layer Feedforward Neural Networks},
  author={Zebin Yang and H. Zhang and A. Sudjianto and Aijun Zhang},
  year={2020}
}
  • Zebin Yang, H. Zhang, +1 author Aijun Zhang
  • Published 2020
  • Computer Science, Mathematics
  • Network initialization is the first and critical step for training neural networks. In this paper, we propose a novel network initialization scheme based on the celebrated Stein's identity. By viewing multi-layer feedforward neural networks as cascades of multi-index models, the projection weights to the first hidden layer are initialized using eigenvectors of the cross-moment matrix between the input's second-order score function and the response. The input data is then forward propagated to… CONTINUE READING

    Topics from this paper.