Information Scaling Law of Deep Neural Networks

@article{Liu2018InformationSL,
  title={Information Scaling Law of Deep Neural Networks},
  author={Xiao-Yang Liu},
  journal={CoRR},
  year={2018},
  volume={abs/1802.04473}
}
With the rapid development of Deep Neural Networks (DNNs), various network models that show strong computing power and impressive expressive power are proposed. However, there is no comprehensive informational interpretation of DNNs from the perspective of information theory. Due to the nonlinear function and the uncertain number of layers and neural units used in the DNNs, the network structure shows nonlinearity and complexity. With the typical DNNs named Convolutional Arithmetic Circuits… CONTINUE READING
Recent Discussions
This paper has been referenced on Twitter 7 times over the past 90 days. VIEW TWEETS