Corpus ID: 220633606

Hybrid Discriminative-Generative Training via Contrastive Learning

@article{Liu2020HybridDT,
  title={Hybrid Discriminative-Generative Training via Contrastive Learning},
  author={Hao Liu and P. Abbeel},
  journal={ArXiv},
  year={2020},
  volume={abs/2007.09070}
}
Contrastive learning and supervised learning have both seen significant progress and success. However, thus far they have largely been treated as two separate objectives, brought together only by having a shared neural network. In this paper we show that through the perspective of hybrid discriminative-generative training of energy-based models we can make a direct connection between contrastive learning and supervised learning. Beyond presenting this unified view, we show our specific choice… Expand
Supervised Contrastive Learning for Pre-trained Language Model Fine-tuning
VAEBM: A Symbiosis between Variational Autoencoders and Energy-based Models
GeDi: Generative Discriminator Guided Sequence Generation
Perfect density models cannot guarantee anomaly detection
Do We Really Need to Learn Representations from In-domain Data for Outlier Detection?
Hybrid Generative-Contrastive Representation Learning
  • Saehoon Kim, Sungwoong Kim, Juho Lee
  • Computer Science
  • ArXiv
  • 2021
PRE-TRAINED LANGUAGE MODEL FINE-TUNING
...
1
2
...

References

SHOWING 1-10 OF 55 REFERENCES
Improved Baselines with Momentum Contrastive Learning
Representation Learning with Contrastive Predictive Coding
Momentum Contrast for Unsupervised Visual Representation Learning
A Simple Framework for Contrastive Learning of Visual Representations
Introspective Neural Networks for Generative Modeling
Introspective Classification with Convolutional Nets
Principled Hybrids of Generative and Discriminative Models
Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination
Learning Generative Models via Discriminative Approaches
  • Zhuowen Tu
  • Computer Science
  • 2007 IEEE Conference on Computer Vision and Pattern Recognition
  • 2007
Image Synthesis with a Single (Robust) Classifier
...
1
2
3
4
5
...