Information-theoretic analysis of generalization capability of learning algorithms

@inproceedings{Xu2017InformationtheoreticAO,
  title={Information-theoretic analysis of generalization capability of learning algorithms},
  author={Aolin Xu and Maxim Raginsky},
  booktitle={NIPS},
  year={2017}
}
We derive upper bounds on the generalization error of a learning algorithm in terms of the mutual information between its input and output. The upper bounds provide theoretical guidelines for striking the right balance between data fit and generalization by controlling the input-output mutual information of a learning algorithm. The results can also be used… CONTINUE READING