Utilizing Geometric Anomalies of High Dimension: When Complexity Makes Computation Easier

@inproceedings{Kainen1997UtilizingGA,
  title={Utilizing Geometric Anomalies of High Dimension: When Complexity Makes Computation Easier},
  author={Paul C. Kainen},
  year={1997}
}
Just as a busy kitchen can be more efficient than an idle one, Kleinrock showed 35 years ago that heavily used networks admit simple heuristic approximations with excellent quantitative accuracy. We describe a number of different examples in which having many parameters actually facilitates computation and we suggest connections with geometric phenomena in high-dimensional spaces. It seems that in several interesting and quite general situations, dimensionality may be a blessing in disguise… 
View via Publisher
34 Citations
Highly Influential Citations
1
Background Citations
21
Methods Citations
2

34 Citations

Limit Theorems as Blessing of Dimensionality: Neural-Oriented Overview

It is shown that such limit theorems often make analysis of complex systems easier—i.e., lead to blessing of dimensionality phenomenon—for all the aspects of these systems: the corresponding transformation, the system’s uncertainty, and the desired result of the system's analysis.

Bringing the Blessing of Dimensionality to the Edge

A distinctive feature of the approach is that, in the supervised setting, the approaches' computational complexity is sub-linear in the number of training samples, which makes it particularly attractive in applications in which the computational power and memory are limited.

Detecting the ultra low dimensionality of real networks

This work proposes a method for detection of the intrinsic dimensionality of high-dimensional networks to reproduce their complex structure using a reduced tractable geometric representation and finds unexpected regularities.

Numerical Methods for Integration and Search for Minima

The stochastic gradient algorithm enabling one to avoid the computation of integrals, hence, the computations of expected values of functions dependent on random variables are described and convergence properties of that algorithm are reported.

Some insights from high-dimensional spheres: Comment on "The unreasonable effectiveness of small neural ensembles in high-dimensional brain" by Alexander N. Gorban et al.

Blessing of dimensionality: mathematical foundations of the statistical physics of data

  • Alexander N GorbanI. Tyukin
  • Mathematics
    Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences
  • 2018
Stochastic separation theorems provide us with classifiers and determine a non-iterative (one-shot) procedure for their construction and allow us to correct legacy artificial intelligence systems.

High-Dimensional Brain in a High-Dimensional World: Blessing of Dimensionality

A brief explanatory review of recent ideas, results and hypotheses about the blessing of dimensionality and related simplifying effects relevant to machine learning and neuroscience is presented.

General stochastic separation theorems with optimal bounds

Blessing of dimensionality at the edge and geometry of few-shot learning

Blessing of dimensionality at the edge

In this paper we present theory and algorithms enabling classes of Artificial Intelligence (AI) systems to continuously and incrementally improve with a-priori quantifiable guarantees - or more…

References

SHOWING 1-10 OF 22 REFERENCES

Sphere Packings, Lattices and Groups

The second edition of this book continues to pursue the question: what is the most efficient way to pack a large number of equal spheres in n-dimensional Euclidean space? The authors also continue to…

Expected Wire Length between Two Randomly Chosen Terminals

Estimates of the Number of Hidden Units and Variation with Respect to Half-Spaces

A geometric method to obtain error-correcting classification by neural networks with fewer hidden units

This work proposes a method for error-correcting classification by feedforward neural networks that reduces the number of hidden units for a variety of classes of patterns.

Quasiorthogonal dimension of euclidean spaces

Some unexpected expected behavior results for bin packing

We study the asymptotic expected behavior of the First Fit and First Fit Decreasing bin packing algorithms applied to items chosen uniformly from the interval (0,u], u ≤ 1. Our results indicate that…

On the immersion of digraphs in cubes

We consider some problems concerning generalizations of embeddings of acyclic digraphs inton-dimensional dicubes. In particular, we define an injectioni from a digraphD into then-dimensional dicubeQn…

Ten lectures on the probabilistic method

This is an examination of what is known about the probabilistic method. Based on the notes from the author's 1986 series of ten lectures, this edition features an additional lecture: The Janson…

Parallel Networks that Learn to Pronounce English Text

H hierarchical clustering techniques applied to NETtalk reveal that these different networks have similar internal representations of letter-to-sound correspondences within groups of processing units, which suggests that invariant internal representations may be found in assemblies of neurons intermediate in size between highly localized and completely distributed representations.

Postural and synergic control for three-dimensional movements of reaching and grasping.

The results suggest that natural movements are mostly carried out in joint space by postural transitions, with the exception of elbow flexion.