Many-Layered Learning

@article{Utgoff2002ManyLayeredL,
  title={Many-Layered Learning},
  author={Paul E. Utgoff and David J. Stracuzzi},
  journal={Neural Computation},
  year={2002},
  volume={14},
  pages={2497-2529}
}
We explore incremental assimilation of new knowledge by sequential learning. Of particular interest is how a network of many knowledge layers can be constructed in an on-line manner, such that the learned units represent building blocks of knowledge that serve to compress the overall representation and facilitate transfer. We motivate the need for many layers of knowledge, and we advocate sequential learning as an avenue for promoting the construction of layered knowledge structures. Finally… Expand
Memory Organization and Knowledge Transfer
TLDR
It is proposed that previously acquired knowledge must be organized, and one possible approach is demonstrated to select the appropriate knowledge structures to transfer when many are available. Expand
On-Line Cumulative Learning of Hierarchical Sparse -grams
TLDR
A novel sparse -gram model that, unlike pruned -grams, learns on-line by stochastic search for frequent -tuple patterns, and is an example of a rare combination—unsupervised, on- line, cumulative, structure learning. Expand
How do We Train Deep Architectures ?
Theoretical results suggest that in order to learn the kind of complicated functions that can represent highlevel abstractions (e.g. in vision, language, and other AI-level tasks), one may need deepExpand
Learning Deep Architectures for AI
TLDR
The motivations and principles regarding learning algorithms for deep architectures, in particular those exploiting as building blocks unsupervised learning of single-layer modelssuch as Restricted Boltzmann Machines, used to construct deeper models such as Deep Belief Networks are discussed. Expand
To go deep or wide in learning?
TLDR
This paper proposes an approach called wide learning based on arc-cosine kernels, that learns a single layer of infinite width and shows that wide learning with single layer outperforms single layer as well as deep architectures of finite width for some benchmark datasets. Expand
Scaling learning algorithms towards AI
TLDR
It is argued that deep architectures have the potential to generalize in non-local ways, i.e., beyond immediate neighbors, and that this is crucial in order to make progress on the kind of complex tasks required for artificial intelligence. Expand
Draft: Deep Learning in Neural Networks: An Overview
TLDR
This historical survey compactly summarises relevant work, much of it from the previous millennium, on deep supervised learning, unsupervised learning, reinforcement learning & evolutionary computation, and indirect search for short programs encoding deep and large networks. Expand
Hierarchical Learning
To build cognitive systems capable of learning complex concepts we need to extend the basic learning models to include learning from previously induced concepts. We outline a broad model whichExpand
Deep learning in neural networks: An overview
TLDR
This historical survey compactly summarizes relevant work, much of it from the previous millennium, review deep supervised learning, unsupervised learning, reinforcement learning & evolutionary computation, and indirect search for short programs encoding deep and large networks. Expand
Incremental Feature Construction for Deep Learning Using Sparse Auto-Encoder
TLDR
A novel alternative method for learning features of each layer of the network by adding primitive/simple features first and then gradually learns finer/more complicated features, which shows that sparse auto-encoders using the in- cremental feature construction provides better accuracy than a sparseAuto-encoder using the conventional feature construction. Expand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 90 REFERENCES
Layered Learning
TLDR
This paper presents layered learning, a hierarchical machine learning paradigm that seamlessly integrates separate learning at each subtask layer, and introduces layered learning in its domain-independent general form. Expand
Using Knowledge to Speed Learning: A Comparison of Knowledge-based Cascade-correlation and Multi-task Learning
TLDR
This paper reports a comparison of KBCC and MTL for their ability to use knowledge to speed learning and to study of the effects of knowledge relevance on learning speed. Expand
Neural network design and the complexity of learning
  • J. S. Judd
  • Computer Science
  • Neural network modeling and connectionism
  • 1990
TLDR
This work formalizes a notion of learning that characterizes the training of feed-forward networks and introduces a perspective on shallow networks, called the Support Cone Interaction graph, which is helpful in distinguishing tractable from intractable subcases. Expand
Learning by Discovering Concept Hierarchies
Abstract We present a new machine learning method that, given a set of training examples, induces a definition of the target concept in terms of a hierarchy of intermediate concepts and theirExpand
Relational Learning for NLP using Linear Threshold Elements
TLDR
A coherent view of learning and reasoning with relational representations in the context of natural language processing is described and extensions of this system along the lines suggested by the theory may provide new levels of scalability and functionality. Expand
A Neuroidal Architecture for Cognitive Computation
TLDR
An architecture is described for designing systems that acquire and manipulate large amounts of unsystematized, or so-called commonsense, knowledge that offers viable approaches to a range of issues that have been previously identified as problematic for artificial intelligence systems that are entirely programmed. Expand
Learning in feedforward layered networks: the tiling algorithm
The authors propose a new algorithm which builds a feedforward layered network in order to learn any Boolean function of N Boolean units. The number of layers and the number of hidden units in eachExpand
Training a 3-node neural network is NP-complete
TLDR
For many simple two-layer networks whose nodes compute linear threshold functions of their inputs that training is NP-complete, it is shown that these networks differ fundamentally from the perceptron in a worst-case computational sense. Expand
Multitask Learning
TLDR
Prior work on MTL is reviewed, new evidence that MTL in backprop nets discovers task relatedness without the need of supervisory signals is presented, and new results for MTL with k-nearest neighbor and kernel regression are presented. Expand
LEARNING CONCEPTS BY ASKING QUESTIONS
TLDR
This chapter describes a program, called Marvin, which uses concepts it has learned previously to learn new concepts, and forms hypotheses about the concept being learned and tests the hypotheses by asking the trainer questions. Expand
...
1
2
3
4
5
...