How Hierarchical Control Self-organizes in Artificial Adaptive Systems

@article{Paine2005HowHC,
  title={How Hierarchical Control Self-organizes in Artificial Adaptive Systems},
  author={Rainer W. Paine and Jun Tani},
  journal={Adaptive Behavior},
  year={2005},
  volume={13},
  pages={211 - 225}
}
Diverse, complex, and adaptive animal behaviors are achieved by organizing hierarchically structured controllers in motor systems. The levels of control progress from simple spinal reflexes and central pattern generators through to executive cognitive control in the frontal cortex. Various types of hierarchical control structures have been introduced and shown to be effective in past artificial agent models, but few studies have shown how such structures can self-organize. This study describes… 
View on SAGE
groups.oist.jp
104 Citations
Highly Influential Citations
5
Background Citations
51
Methods Citations
20
Results Citations
9

Figures from this paper

104 Citations

Citation Type

Citation Type

Self-organizing high-order cognitive functions in artificial agents: Implications for possible prefrontal cortex mechanisms
Emergence of Functional Hierarchy in a Multiple Timescale Neural Network Model: A Humanoid Robot Experiment
TLDR
The proposed network model, coordinating the physical body of a humanoid robot through high-dimensional sensori-motor control, also successfully situated itself within a physical environment and suggests that it is not only the spatial connections between neurons but also the timescales of neural activity that act as important mechanisms leading to functional hierarchy in neural systems.
A Hierarchical Autonomous Robot Controller for Learning and Memory: Adaptation in a Dynamic Environment
TLDR
Experimental results show that the proposed model can produce a robot with learning and memorizing capabilities that enable it to survive in complex and highly dynamic environments.
Acquiring Rules for Rules: Neuro-Dynamical Systems Account for Meta-Cognition
TLDR
This study employs evolutionary processes to explore neuronal mechanisms accounting for meta-level rule switching and shows that neural network division into a lower sensorimotor level and a higher cognitive level enhances the performance of the robot in the given tasks.
Learning programs is better than learning dynamics: A programmable neural network hierarchical architecture in a multi-task scenario
TLDR
This paper advances a novel proposal, a hierarchical programmable neural network architecture, based on the notion of programmability and an interpreter-programmer computational scheme that provides a robust, scalable and flexible scheme that can be iterated at different hierarchical layers permitting to learn, encode and control multiple qualitatively different behaviors.
Achieving "organic compositionality" through self-organization: Reviews on brain-inspired robotics experiments
Ego-centric and allo-centric abstraction in self-organized hierarchical neural networks
TLDR
This work investigates effective assemblies for simultaneously placing ego-centric and allo-centric processes in the cognitive hierarchy, by evolving self-organized neural network controllers, by considering how they are related with known assumptions about human brain functionality.
Learning to Achieve Different Levels of Adaptability for Human–Robot Collaboration Utilizing a Neuro-Dynamical System
TLDR
Experimental results show that the robot can perform collaborative tasks under dynamically changing environments, including both learned and unlearned situations, thanks to different levels of adaptability acquired in the system.
Behavioral control through evolutionary neurocontrollers for autonomous mobile robot navigation
Evolution of Signaling in a Multi-Robot System: Categorization and Communication
TLDR
This article describes a set of experiments in which artificial evolution is used as a means to engineer robot neuro-controllers capable of guiding groups of robots in a categorization task by producing appropriate actions.
...
...

References

SHOWING 1-10 OF 90 REFERENCES
Motor primitive and sequence self-organization in a hierarchical recurrent neural network
Emergence of Memory-Driven Command Neurons in Evolved Artificial Agents
TLDR
The accessibility of evolved ANNs for a detailed inspection, together with the resemblance of some of the results to known findings from neurobiology, places evolve ANNs as an excellent candidate model for the study of structure and function relationship in complex nervous systems.
Learning to generate articulated behavior through the bottom-up and the top-down interaction processes
  • J. Tani
  • Computer Science, Psychology
    Neural Networks
  • 2003
Learning to perceive the world as articulated: an approach for hierarchical learning in sensory-motor systems
Automatic creation of an autonomous agent: genetic evolution of a neural-network driven robot
TLDR
The paper describes the results of the evolutionary development of a real, neural-network driven mobile robot, and shows a number of emergent phenomena that are characteristic of autonomous agents.
Evolving robots able to self-localize in the environment: the importance of viewing cognition as the result of processes occurring at different time-scales
TLDR
It is shown how autonomous robots synthesized through an evolutionary process can solve problems that necessarily require an ability to integrate sensorimotor information over time.
An Evolutionary Ecological Approach to the Study of Learning Behavior Using a Robot-Based Model
TLDR
This article explores the applicability of integrated neural networks with fixed connection weights and simple 'leaky-integrator' neurons as controllers for autonomous learning robots and shows that such a control system is capable of integrating reactive and learned behaviour without explicitly needing hand-designed modules, dedicated to a particular behavior, or an externally introduced reinforcement signal.
Levels of dynamics and adaptive behavior in evolutionary neural controllers
TLDR
Two classes of dynamical recurrent neural networks are compared on two behavioral tasks aimed at exploring their capabilities to display reinforcement-learning like behaviors and adaptation to unpredictable environmental changes, but PNNs display significantly better performance when sensory-motor re-adaptation is required after the evolutionary process.
Neuromodulation of Reactive Sensorimotor Mappings as a Short-Term Memory Mechanism in Delayed Response Tasks
TLDR
This article demonstrates and discusses the use of synaptic plasticity, realized through neuromodulation of sensorimotor mappings, as a short-term memory mechanism in delayed response tasks with extended sequential cascaded networks.
Learning of action through adaptive combination of motor primitives
TLDR
This work shows that humans learn the dynamics of reaching movements through a flexible combination of primitives that have gaussian-like tuning functions encoding hand velocity, and finds close agreement between the predicted limitations and the subjects’ adaptation to new force fields.
...
...