Optimization landscape in the simplest constrained random least-square problem

@article{Fyodorov2022OptimizationLI,
  title={Optimization landscape in the simplest constrained random least-square problem},
  author={Yan V. Fyodorov and Rashel Tublin},
  journal={Journal of Physics A: Mathematical and Theoretical},
  year={2022},
  volume={55}
}
  • Y. Fyodorov, Rashel Tublin
  • Published 26 December 2021
  • Mathematics, Computer Science
  • Journal of Physics A: Mathematical and Theoretical
We analyze statistical features of the ‘optimization landscape’ in a random version of one of the simplest constrained optimization problems of the least-square type: finding the best approximation for the solution of a system of M linear equations in N unknowns: ( a k , x ) = b k , k = 1, …, M on the N-sphere x 2 = N. We treat both the N-component vectors a k and parameters b k as independent mean zero real Gaussian random variables. First, we derive the exact expressions for the mean number… 

Superposition of random plane waves in high spatial dimensions: Random matrix approach to landscape complexity

Motivated by current interest in understanding statistical properties of random landscapes in high-dimensional spaces, we consider a model of the landscape in [Formula: see text] obtained by

The high-d landscapes paradigm: spin-glasses, and beyond

In this Chapter we review recent developments on the characterization of random landscapes in high-dimension. We focus in particular on the problem of characterizing the landscape topology and

References

SHOWING 1-10 OF 48 REFERENCES

Topology Trivialization and Large Deviations for the Minimum in the Simplest Random Optimization

Finding the global minimum of a cost function given by the sum of a quadratic and a linear form in N real variables over (N−1)-dimensional sphere is one of the simplest, yet paradigmatic problems in

Complexity of random energy landscapes, glass transition, and absolute value of the spectral determinant of random matrices.

Finding the mean of the total number N(tot) of stationary points for N-dimensional random energy landscapes is reduced to averaging the absolute value of the characteristic polynomial of the

Landscape Complexity for the Empirical Risk of Generalized Linear Models

TLDR
An explicit variational formula is obtained for the quenched complexity, which is generally different from its annealed counterpart, and allows to obtain the number of critical points for typical instances up to exponential accuracy.

Replica Symmetry Breaking Condition Exposed by Random Matrix Calculation of Landscape Complexity

Abstract We start with a rather detailed, general discussion of recent results of the replica approach to statistical mechanics of a single classical particle placed in a random N(≫1)-dimensional

Large deviations of extreme eigenvalues of random matrices.

TLDR
The average density of states in matrices whose eigenvalues are restricted to be larger than a fixed number zeta is calculated, thus generalizing the celebrated Wigner semicircle law.

The averaged characteristic polynomial for the Gaussian and chiral Gaussian ensembles with a source

TLDR
The Dyson Brownian motion model is used to give a meaning for general β > 0, and new derivations of the general case of Desrosiers' dualities are given.

Critical behavior of the number of minima of a random landscape at the glass transition point and the Tracy-Widom distribution.

We exploit a relation between the mean number N(m) of minima of random Gaussian surfaces and extreme eigenvalues of random matrices to understand the critical behavior of N(m) in the simplest

Random Matrices and Complexity of Spin Glasses

TLDR
This study enables detailed information about the bottom of the energy landscape, including the absolute minimum, and the other local minima, and describes an interesting layered structure of the low critical values for the Hamiltonians of these models.

Complex Energy Landscapes in Spiked-Tensor and Simple Glassy Models: Ruggedness, Arrangements of Local Minima, and Phase Transitions

TLDR
This work develops a framework based on the Kac-Rice method that allows to compute the complexity of the landscape, i.e. the logarithm of the typical number of stationary points and their Hessian, and discusses its advantages with respect to previous frameworks.