Corpus ID: 220633606

Hybrid Discriminative-Generative Training via Contrastive Learning

@article{Liu2020HybridDT,
  title={Hybrid Discriminative-Generative Training via Contrastive Learning},
  author={Hao Liu and Pieter Abbeel},
  journal={ArXiv},
  year={2020},
  volume={abs/2007.09070}
}
  • Hao Liu, Pieter Abbeel
  • Published 2020
  • Computer Science, Mathematics
  • ArXiv
  • Contrastive learning and supervised learning have both seen significant progress and success. However, thus far they have largely been treated as two separate objectives, brought together only by having a shared neural network. In this paper we show that through the perspective of hybrid discriminative-generative training of energy-based models we can make a direct connection between contrastive learning and supervised learning. Beyond presenting this unified view, we show our specific choice… CONTINUE READING

    Figures, Tables, and Topics from this paper.

    Explore Further: Topics Discussed in This Paper

    References

    Publications referenced by this paper.
    SHOWING 1-10 OF 48 REFERENCES

    Learning Multiple Layers of Features from Tiny Images

    VIEW 8 EXCERPTS
    HIGHLY INFLUENTIAL

    Bayesian Learning via Stochastic Gradient Langevin Dynamics

    VIEW 13 EXCERPTS
    HIGHLY INFLUENTIAL

    Deep Learning Face Attributes in the Wild

    VIEW 5 EXCERPTS
    HIGHLY INFLUENTIAL

    On Calibration of Modern Neural Networks

    VIEW 6 EXCERPTS
    HIGHLY INFLUENTIAL

    Reading Digits in Natural Images with Unsupervised Feature Learning

    VIEW 4 EXCERPTS
    HIGHLY INFLUENTIAL

    Supervised Contrastive Learning

    VIEW 12 EXCERPTS
    HIGHLY INFLUENTIAL

    Your Classifier is Secretly an Energy Based Model and You Should Treat it Like One

    VIEW 10 EXCERPTS
    HIGHLY INFLUENTIAL

    Glow: Generative Flow with Invertible 1x1 Convolutions

    VIEW 4 EXCERPTS
    HIGHLY INFLUENTIAL

    Implicit Generation and Modeling with Energy Based Models

    VIEW 8 EXCERPTS
    HIGHLY INFLUENTIAL