# Statistical physics of inference: thresholds and algorithms

@article{Zdeborov2015StatisticalPO, title={Statistical physics of inference: thresholds and algorithms}, author={Lenka Zdeborov{\'a} and Florent Krzakala}, journal={Advances in Physics}, year={2015}, volume={65}, pages={453 - 552} }

Many questions of fundamental interest in today's science can be formulated as inference problems: some partial, or noisy, observations are performed over a set of variables and the goal is to recover, or infer, the values of the variables based on the indirect information contained in the measurements. For such problems, the central scientific questions are: Under what conditions is the information contained in the measurements sufficient for a satisfactory inference to be possible? What are…

## 294 Citations

### High-dimensional inference: a statistical mechanics perspective

- Computer ScienceArXiv
- 2020

This article aims at emphasizing some of the deep links connecting these apparently separated disciplines through the description of paradigmatic models of high-dimensional inference in the language of statistical mechanics.

### Statistical mechanics approaches to optimization and inference

- Computer Science
- 2017

This thesis will propose several statistical mechanics based models able to deal with two types of problems: optimization and inference problems and a set of combinatorial optimization problems on graphs, the Prize collecting and the Packing of Steiner trees problems.

### Understanding Phase Transitions via Mutual Information and MMSE

- Computer ScienceInformation-Theoretic Methods in Data Science
- 2021

This chapter presents a tutorial description of the standard linear model and its connection to information theory, and describes the replica prediction for this model and outlines the authors' recent proof that it is exact.

### Inverse statistical problems: from the inverse Ising problem to data science

- Mathematics
- 2017

This review focuses on the inverse Ising problem and closely related problems, namely how to infer the coupling strengths between spins given observed spin correlations, magnetizations, or other data.

### On the glassy nature of the hard phase in inference problems

- Computer SciencePhysical Review X
- 2019

This work shows that for noise-to-signal ratios that are below the information theoretic threshold, the posterior measure is composed of an exponential number of metastable glassy states and it is shown that this glassiness extends even slightly below the algorithmic threshold below which the well-known approximate message passing algorithm is able to closely reconstruct the signal.

### Mean- eld theory of high-dimensional Bayesian inference

- Computer Science
- 2020

The goal is to recall basic (deep) concepts, as well as to provide some modern analytic and algorithmic tools used in high-dimensional inference.

### Correspondence between thermodynamics and inference.

- Computer SciencePhysical review. E
- 2019

It is proposed that the Gibbs entropy provides a natural device for counting distinguishable distributions in the context of Bayesian inference and is used to define a generalized principle of indifference in which every distinguishable model is assigned equal a priori probability.

### Fundamental problems in Statistical Physics XIV: Lecture on Machine Learning

- Computer SciencePhysica A: Statistical Mechanics and its Applications
- 2022

### Typology of phase transitions in Bayesian inference problems

- Computer SciencePhysical review. E
- 2019

The results show that the instability of the trivial fixed point is not generic evidence for the Bayes optimality of the message-passing algorithms, and shed light on the existence of hybrid-hard phases for a large class of planted constraint satisfaction problems.

### Statistical physics of linear and bilinear inference problems

- Computer Science
- 2016

The aim of this thesis is to propose efficient algorithms for matrix compressed sensing and to perform their theoretical analysis, which reveals phases in which inference is easy, hard or impossible and of instabilities in Bayesian bilinear inference algorithms.

## References

SHOWING 1-10 OF 435 REFERENCES

### Understanding belief propagation and its generalizations

- Computer Science
- 2003

It is shown that BP can only converge to a fixed point that is also a stationary point of the Bethe approximation to the free energy, which enables connections to be made with variational approaches to approximate inference.

### Statistical Physics of Hard Optimization Problems

- Computer ScienceArXiv
- 2008

A new class of problems is introduced which is named "locked" constraint satisfaction, where the statistical description is easily solvable, but from the algorithmic point of view they are even more challenging than the canonical satisfiability.

### Advanced mean field methods: theory and practice

- Computer Science
- 2001

The theoretical foundations of advanced mean field methods are covered, the relation between the different approaches are explored, the quality of the approximation obtained is examined, and their application to various areas of probabilistic modeling is demonstrated.

### Statistical mechanics of learning from examples.

- Computer SciencePhysical review. A, Atomic, molecular, and optical physics
- 1992

It is shown that for smooth networks, i.e., those with continuously varying weights and smooth transfer functions, the generalization curve asymptotically obeys an inverse power law, while for nonsmooth networks other behaviors can appear, depending on the nature of the nonlinearities as well as the realizability of the rule.

### The Elements of Statistical Learning: Data Mining, Inference, and Prediction

- Computer Science
- 2004

This book is a valuable resource, both for the statistician needing an introduction to machine learning and related elds and for the computer scientist wishing to learn more about statistics, and statisticians will especially appreciate that it is written in their own language.

### The Nature of Computation

- Computer Science, Art
- 2011

The authors explain why the P vs. NP problem is so fundamental, and why it is so hard to resolve, and lead the reader through the complexity of mazes and games; optimization in theory and practice; randomized algorithms, interactive proofs, and pseudorandomness; Markov chains and phase transitions; and the outer reaches of quantum computing.

### Graphical Models, Exponential Families, and Variational Inference

- Computer ScienceFound. Trends Mach. Learn.
- 2008

The variational approach provides a complementary alternative to Markov chain Monte Carlo as a general source of approximation methods for inference in large-scale statistical models.

### Probabilistic Inference Using Markov Chain Monte Carlo Methods

- Computer Science
- 2011

The role of probabilistic inference in artificial intelligence is outlined, the theory of Markov chains is presented, and various Markov chain Monte Carlo algorithms are described, along with a number of supporting techniques.

### Statistical mechanics of unsupervised structure recognition

- Computer Science
- 1994

A model of unsupervised learning is studied, where the environment provides N-dimensional input examples that are drawn from two overlapping Gaussian clouds and how well the underlying structure is inferred from a set of examples is investigated.

### Statistical mechanics of community detection.

- Computer SciencePhysical review. E, Statistical, nonlinear, and soft matter physics
- 2006

The properties of the ground state configuration are elucidated to give a concise definition of communities as cohesive subgroups in networks that is adaptive to the specific class of network under study.