Comparing Information-Theoretic Measures of Complexity in Boltzmann Machines

@article{Kanwal2017ComparingIM,
  title={Comparing Information-Theoretic Measures of Complexity in Boltzmann Machines},
  author={Maxinder S. Kanwal and Joshua A. Grochow and N. Ay},
  journal={Entropy},
  year={2017},
  volume={19},
  pages={310}
}
  • Maxinder S. Kanwal, Joshua A. Grochow, N. Ay
  • Published 2017
  • Computer Science, Mathematics, Biology
  • Entropy
  • In the past three decades, many theoretical measures of complexity have been proposed to help understand complex systems. In this work, for the first time, we place these measures on a level playing field, to explore the qualitative similarities and differences between them, and their shortcomings. Specifically, using the Boltzmann machine architecture (a fully connected recurrent neural network) with uniformly distributed weights as our model of study, we numerically measure how complexity… CONTINUE READING
    18 Citations
    Information integration in large brain networks
    • 20
    • PDF
    Complexity as Causal Information Integration
    • PDF
    A general spectral decomposition of causal influences applied to integrated information
    • 1
    Degeneracy and Redundancy in Active Inference.
    • 5
    Tangent Space Sensitivity and Distribution of Linear Regions in ReLU Networks
    • PDF
    Tangent Space Separability in Feedforward Neural Networks
    • 1
    • PDF

    References

    SHOWING 1-10 OF 29 REFERENCES
    Predictability , Complexity , and Learning
    • 335
    • PDF
    On complexity measures
    • 13
    Regularities unseen, randomness observed: levels of entropy convergence.
    • 327
    • PDF
    Toward a quantitative theory of self-generated complexity
    • 569
    • PDF
    Practical Measures of Integrated Information for Time-Series Data
    • 141
    • PDF
    Complex adaptive systems - an introduction to computational models of social life
    • J. Miller, S. Page
    • Computer Science, Sociology
    • Princeton studies in complexity
    • 2007
    • 1,599
    • PDF
    Integrated Information Increases with Fitness in the Evolution of Animats
    • 97
    • PDF
    Complex Systems Theory
    • 22
    • PDF
    A measure for brain complexity: relating functional segregation and integration in the nervous system.
    • 1,243
    • PDF
    Introduction to the theory of neural computation
    • 6,034
    • PDF