Corpus ID: 235125543

ReduNet: A White-box Deep Network from the Principle of Maximizing Rate Reduction

@article{Chan2021ReduNetAW,
  title={ReduNet: A White-box Deep Network from the Principle of Maximizing Rate Reduction},
  author={Kwan Ho Ryan Chan and Yaodong Yu and Chong You and Haozhi Qi and John Wright and Yi Ma},
  journal={ArXiv},
  year={2021},
  volume={abs/2105.10446}
}
This work attempts to provide a plausible theoretical framework that aims to interpret modern deep (convolutional) networks from the principles of data compression and discriminative representation. We argue that for high-dimensional multi-class data, the optimal linear discriminative representation maximizes the coding rate difference between the whole dataset and the average of all the subsets. We show that the basic iterative gradient ascent scheme for optimizing the rate reduction objective… Expand
1 Citations