Corpus ID: 235446430

Nonequilibrium thermodynamics of self-supervised learning

@article{Salazar2021NonequilibriumTO,
  title={Nonequilibrium thermodynamics of self-supervised learning},
  author={D. S. P. Salazar},
  journal={ArXiv},
  year={2021},
  volume={abs/2106.08981}
}
Self-supervised learning (SSL) of energy based models has an intuitive relation to equilibrium thermodynamics because the softmax layer, mapping energies to probabilities, is a Gibbs distribution. However, in what way SSL is a thermodynamic process? We show that some SSL paradigms behave as a thermodynamic composite system formed by representations and self-labels in contact with a nonequilibrium reservoir. Moreover, this system is subjected to usual thermodynamic cycles, such as adiabatic… Expand

Figures from this paper

References

SHOWING 1-10 OF 34 REFERENCES
PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES.
TLDR
Since direct observational evidence for the existence of a cometary belt may not be available for some time to come, it is necessary to look for possible gravitational effects. Expand
"J."
however (for it was the literal soul of the life of the Redeemer, John xv. io), is the peculiar token of fellowship with the Redeemer. That love to God (what is meant here is not God’s love to men)Expand
A
  • Vedaldi, International Conference on Learning Representations (ICLR)
  • 2020
and A
  • Joulin. In Proceedings of Advances in Neural Information Processing Systems (NeurIPS)
  • 2020
C
  • ́ Tallec, P. Richemond, E. Buchatskaya, C. Doersch, B. Pires, Z. Guo, M. Azar, et al. In Proceedings of Advances in Neural Information Processing Systems (NeurIPS)
  • 2020
K
  • He and P. Dollar, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
  • 2020
Rev
  • Lett. 124, 020503
  • 2020
S
  • Xie and R. Girshick, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
  • 2020
Rev
  • Lett. 122, 080602
  • 2019
Rev
  • Mod. Phys. 91, 045002
  • 2019
...
1
2
3
4
...