A Capacity Scaling Law for Artificial Neural Networks

@article{Friedland2017ACS,
  title={A Capacity Scaling Law for Artificial Neural Networks},
  author={Gerald Friedland and Mario Michael Krell},
  journal={CoRR},
  year={2017},
  volume={abs/1708.06019}
}
In this article, we derive the calculation of two critical numbers that quantify the capabilities of artificial neural networks with gating functions, such as sign, sigmoid, or rectified linear units. First, we derive the calculation of the Vapnik–Chervonenkis (VC) dimension of a network with binary output layer, which is the theoretical limit for perfect fitting of the training data. Second, we derive what we call the MacKay dimension of the network. This is a theoretical limit indicating… CONTINUE READING