Convergence of Backpropagation with Momentum for Network Architectures with Skip Connections

@article{Agarwal2017ConvergenceOB,
  title={Convergence of Backpropagation with Momentum for Network Architectures with Skip Connections},
  author={Chirag Agarwal and Joe Klobusicky and Dan Schonfeld},
  journal={arXiv: Computer Vision and Pattern Recognition},
  year={2017}
}
We study a class of deep neural networks with networks that form a directed acyclic graph (DAG). For backpropagation defined by gradient descent with adaptive momentum, we show weights converge for a large class of nonlinear activation functions. The proof generalizes the results of Wu et al. (2008) who showed convergence for a feed forward network with one hidden layer. For an example of the effectiveness of DAG architectures, we describe an example of compression through an autoencoder, and… Expand
1 Citations