Corpus ID: 18824582

Evolving Culture vs Local Minima

@article{Bengio2012EvolvingCV,
  title={Evolving Culture vs Local Minima},
  author={Yoshua Bengio},
  journal={ArXiv},
  year={2012},
  volume={abs/1203.2990}
}
  • Yoshua Bengio
  • Published 2012
  • Computer Science
  • ArXiv
  • We propose a theory that relates difficulty of learning in deep architectures to culture and language. It is articulated around the following hypotheses: (1) learning in an individual human brain is hampered by the presence of effective local minima; (2) this optimization difficulty is particularly important when it comes to learning higher-level abstractions, i.e., concepts that cover a vast and highly-nonlinear span of sensory configurations; (3) such high-level abstractions are best… CONTINUE READING

    Figures and Topics from this paper.

    Paper Mentions

    Deep Learning of Representations
    • 55
    • PDF
    Knowledge Matters: Importance of Prior Information for Optimization
    • 114
    • PDF
    Deep Learning of Representations: Looking Forward
    • 402
    • PDF
    Computational Foundations of Natural Intelligence
    • 12
    • PDF
    Computational Foundations of Natural Intelligence
    Computational Foundations of Natural Intelligence
    • 12
    • PDF
    Computational Foundations of Natural Intelligence
    • 1
    Raising AI: Tutoring Matters
    • 9
    • PDF

    References

    Publications referenced by this paper.
    SHOWING 1-10 OF 52 REFERENCES
    Learning Deep Architectures for AI
    • 6,172
    • PDF
    Scaling learning algorithms towards AI
    • 953
    • PDF
    Curriculum learning
    • 1,989
    • PDF
    Greedy Layer-Wise Training of Deep Networks
    • 1,598
    • Highly Influential
    Connectionist Learning Procedures
    • 1,419
    • PDF
    Learning and development in neural networks: the importance of starting small
    • 1,483
    • PDF
    Exploring Strategies for Training Deep Neural Networks
    • 878
    • PDF
    Efficient Learning of Deep Boltzmann Machines
    • 296
    • PDF
    Flexible shaping: How learning in small steps helps
    • 112
    • PDF
    A Fast Learning Algorithm for Deep Belief Nets
    • 10,956
    • Highly Influential
    • PDF