# Singularities Affect Dynamics of Learning in Neuromanifolds

@article{Amari2006SingularitiesAD, title={Singularities Affect Dynamics of Learning in Neuromanifolds}, author={Shun-ichi Amari and Hyeyoung Park and Tomoko Ozeki}, journal={Neural Computation}, year={2006}, volume={18}, pages={1007-1065} }

- Published 2006 in Neural Computation
DOI:10.1162/neco.2006.18.5.1007

The parameter spaces of hierarchical systems such as multilayer perceptrons include singularities due to the symmetry and degeneration of hidden units. A parameter space forms a geometrical manifold, called the neuromanifold in the case of neural networks. Such a model is identified with a statistical model, and a Riemannian metric is given by the Fisher information matrix. However, the matrix degenerates at singularities. Such a singular structure is ubiquitous not only in multilayer… CONTINUE READING

#### Citations

##### Publications citing this paper.

Showing 1-10 of 49 extracted citations

## Parameter Identifiability in Statistical Machine Learning: A Review

View 20 Excerpts

Highly Influenced

## Skip Connections as Effective Symmetry-Breaking

View 7 Excerpts

Highly Influenced

## Learning Dynamics of a Single Polar Variable Complex-Valued Neuron

View 4 Excerpts

Highly Influenced

## Numerical Analysis near Singularities in RBF Networks

View 4 Excerpts

Highly Influenced

## Information Geometry and Minimum Description Length Networks

View 4 Excerpts

Highly Influenced

## Analysis of Learning Near Temporary Minima Using the All Permutations Fuzzy Rule-Base

View 3 Excerpts

Highly Influenced

## Reducing Duplicate Filters in Deep Neural Networks

View 2 Excerpts

#### Citation Statistics

#### 134 Citations

Citations per Year

Semantic Scholar estimates that this publication has

**134**citations based on the available data.See our **FAQ** for additional information.

#### References

##### Publications referenced by this paper.

Showing 1-10 of 55 references

## Likelihood Ratio of Unidentifiable Models and Multilayer Neural Networks

View 8 Excerpts

Highly Influenced

## Estimating the dimension of a linear model

View 5 Excerpts

Highly Influenced

## New consideration on criteria of model selection

View 6 Excerpts

Highly Influenced

## A failure of likelihood asymptotics for normal mixtures

View 8 Excerpts

Highly Influenced

## Testing in Locally Conic Models and Application to Mixture Models

View 7 Excerpts

Highly Influenced

## Asymptotics for likelihood ratio tests under loss of identifiability

View 3 Excerpts

Highly Influenced

## Differential and algebraic geometry of multilayer perceptrons

View 5 Excerpts

Highly Influenced

## Natural gradient descent for on-line learning

View 4 Excerpts

Highly Influenced

## Difficulty of Singularity in Population Coding

View 1 Excerpt