Nonequilibrium thermodynamics of self-supervised learning

@article{Salazar2021NonequilibriumTO,
  title={Nonequilibrium thermodynamics of self-supervised learning},
  author={Domingos S. P. Salazar},
  journal={ArXiv},
  year={2021},
  volume={abs/2106.08981}
}

Figures from this paper

Detecting Cybersecurity Attacks in Internet of Things Using Artificial Intelligence Methods: A Systematic Literature Review

TLDR
A systematic literature review that categorize, map and survey the existing literature on AI methods used to detect cybersecurity attacks in the IoT environment and provides an insight into the AI roadmap to detect threats based on attack categories is presented.

References

SHOWING 1-10 OF 39 REFERENCES

Thermodynamic efficiency of learning a rule in neural networks

TLDR
Using stochastic thermodynamics, it is shown that the thermodynamic costs of the learning process provide an upper bound on the amount of information that the network is able to learn from its teacher for both batch and online learning.

Spectral dynamics of learning in restricted Boltzmann machines

TLDR
A generic statistical ensemble is proposed for the weight matrix of the RBM and its mean evolution is characterized, unveiling in some way how the selected modes interact in later stages of the learning procedure and defining a deterministic learning curve for the R BM.

Entropy production fluctuation theorem and the nonequilibrium work relation for free energy differences.

  • G. Crooks
  • Economics, Physics
    Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics
  • 1999
TLDR
A generalized version of the fluctuation theorem is derived for stochastic, microscopically reversible dynamics and this generalized theorem provides a succinct proof of the nonequilibrium work relation.

Stochastic thermodynamics: principles and perspectives

Stochastic thermodynamics provides a framework for describing small systems like colloids or biomolecules driven out of equilibrium but still in contact with a heat bath. Both, a first-law like

Deep autoregressive models for the efficient variational simulation of many-body quantum systems

TLDR
This work proposes a specialized neural- network architecture that supports efficient and exact sampling, completely circumventing the need for Markov-chain sampling, and demonstrates the ability to obtain accurate results on larger system sizes than those currently accessible to neural-network quantum states.

Machine learning and the physical sciences

TLDR
This article reviews in a selective way the recent research on the interface between machine learning and the physical sciences, including conceptual developments in ML motivated by physical insights, applications of machine learning techniques to several domains in physics, and cross fertilization between the two fields.

Solving the quantum many-body problem with artificial neural networks

TLDR
A variational representation of quantum states based on artificial neural networks with a variable number of hidden neurons and a reinforcement-learning scheme that is capable of both finding the ground state and describing the unitary time evolution of complex interacting quantum systems.

Inverse statistical problems: from the inverse Ising problem to data science

TLDR
This review focuses on the inverse Ising problem and closely related problems, namely how to infer the coupling strengths between spins given observed spin correlations, magnetizations, or other data.

Learning physical properties of liquid crystals with deep convolutional neural networks

TLDR
By optimizing simple architectures, this work finds that convolutional neural networks can predict physical properties of liquid crystals with exceptional accuracy and shows that these deep neural networks identify liquid crystal phases and predict the order parameter of simulated nematic liquid crystals almost perfectly.

Optimal errors and phase transitions in high-dimensional generalized linear models

TLDR
R rigorously establishes rigorously the intrinsic information-theoretic limitations of inference and learning for a class of randomly generated instances of generalized linear models, thus closing several decades-old conjectures.