Exact information propagation through fully-connected feed forward neural networks

@article{Burkholz2018ExactIP,
  title={Exact information propagation through fully-connected feed forward neural networks},
  author={Rebekka Burkholz and Alina Dubatovka},
  journal={CoRR},
  year={2018},
  volume={abs/1806.06362}
}
Neural network ensembles at initialisation give rise to the trainability and training speed of neural networks and thus support parameter choices at initialisation. These insights rely so far on mean field approximations that assume infinite layer width and study average squared signals. Thus, information about the full output distribution gets lost… CONTINUE READING