Corpus ID: 209370467

General Information Bottleneck Objectives and their Applications to Machine Learning

@article{Mukherjee2019GeneralIB,
  title={General Information Bottleneck Objectives and their Applications to Machine Learning},
  author={Sayandev Mukherjee},
  journal={ArXiv},
  year={2019},
  volume={abs/1912.06248}
}
  • Sayandev Mukherjee
  • Published in ArXiv 2019
  • Mathematics, Computer Science
  • We view the Information Bottleneck Principle (IBP: Tishby et al., 1999; Schwartz-Ziv and Tishby, 2017) and Predictive Information Bottleneck Principle (PIBP: Still et al., 2007; Alemi, 2019) as special cases of a family of general information bottleneck objectives (IBOs). Each IBO corresponds to a particular constrained optimization problem where the constraints apply to: (a) the mutual information between the training data and the learned model parameters or extracted representation of the… CONTINUE READING

    Citations

    Publications citing this paper.

    References

    Publications referenced by this paper.
    SHOWING 1-10 OF 13 REFERENCES

    Variational Predictive Information Bottleneck

    VIEW 21 EXCERPTS
    HIGHLY INFLUENTIAL

    Information-theoretic analysis of generalization capability of learning algorithms

    VIEW 8 EXCERPTS
    HIGHLY INFLUENTIAL

    The information bottleneck method

    VIEW 5 EXCERPTS
    HIGHLY INFLUENTIAL

    Optimal Information Processing and Bayes's Theorem

    VIEW 4 EXCERPTS
    HIGHLY INFLUENTIAL

    Tightening Mutual Information Based Bounds on Generalization Error

    VIEW 3 EXCERPTS