Statistical Physics and Practical Training of Soft – Committee Machines

@inproceedings{Ahr1998StatisticalPA,
title={Statistical Physics and Practical Training of Soft – Committee Machines},
author={Martin Ahr and Michael Biehl and Robert Urbanczik},
year={1998}
}

Equilibrium states of large layered neural networks with differentiable activation function and a single, linear output unit are investigated using the replica formalism. The quenched free energy of a student network with a very large number of hidden units learning a rule of perfectly matching complexity is calculated analytically. The system undergoes a first order phase transition from unspecialized to specialized student configurations at a critical size of the training set. Computer… CONTINUE READING