Corpus ID: 53730466

Autonomous Extraction of a Hierarchical Structure of Tasks in Reinforcement Learning, A Sequential Associate Rule Mining Approach

@article{Ghazanfari2018AutonomousEO,
  title={Autonomous Extraction of a Hierarchical Structure of Tasks in Reinforcement Learning, A Sequential Associate Rule Mining Approach},
  author={Behzad Ghazanfari and Fatemeh Afghah and Matthew E. Taylor},
  journal={ArXiv},
  year={2018},
  volume={abs/1811.08275}
}
Reinforcement learning (RL) techniques, while often powerful, can suffer from slow learning speeds, particularly in high dimensional spaces. Decomposition of tasks into a hierarchical structure holds the potential to significantly speed up learning, generalization, and transfer learning. However, the current task decomposition techniques often rely on high-level knowledge provided by an expert (e.g. using dynamic Bayesian networks) to extract a hierarchical task structure; which is not… Expand
2 Citations
From Reinforcement Learning Towards Artificial General Intelligence
TLDR
It is proposed that this road is breaking through barriers in Reinforcement Learning and making us closer to Artificial General Intelligence, and views about the current challenges to get us further towards this goal are shared. Expand
Wildfire Monitoring in Remote Areas using Autonomous Unmanned Aerial Vehicles
TLDR
Simulation results confirm that the performance of the proposed system- without the need for inter-coalition communications- approaches that of a centrally-optimized system. Expand

References

SHOWING 1-10 OF 28 REFERENCES
Discovering hierarchy in reinforcement learning
TLDR
This thesis addresses the open problem of automatically discovering hierarchical structure in reinforcement learning with an algorithm, called HEXQ, that automatically decomposes and solves a multi-dimensional Markov decision problem (MDP) by constructing aMulti-level hierarchy of interlinked subtasks without being given the model beforehand. Expand
Automatic Discovery and Transfer of Task Hierarchies in Reinforcement Learning
TLDR
It is demonstrated that causally motivated task hierarchies transfer more robustly than other kinds of detailed knowledge that depend on the idiosyncrasies of the source domain and are hence less transferable. Expand
Automatic discovery and transfer of MAXQ hierarchies
TLDR
It is demonstrated empirically that HI-MAT constructs compact hierarchies that are comparable to manually-engineered hierarchies and facilitate significant speedup in learning when transferred to a target task. Expand
Hierarchical Reinforcement Learning with the MAXQ Value Function Decomposition
TLDR
The paper presents an online model-free learning algorithm, MAXQ-Q, and proves that it converges with probability 1 to a kind of locally-optimal policy known as a recursively optimal policy, even in the presence of the five kinds of state abstraction. Expand
Recent Advances in Hierarchical Reinforcement Learning
TLDR
This work reviews several approaches to temporal abstraction and hierarchical organization that machine learning researchers have recently developed and discusses extensions of these ideas to concurrent activities, multiagent coordination, and hierarchical memory for addressing partial observability. Expand
Extracting bottlenecks for reinforcement learning agent by holonic concept clustering and attentional functions
TLDR
Holonic concept clustering and attentional functions are proposed to extract bottleneck states by means of a holonic clustering to extract high-level concepts and the proposed mechanism has a better time complexity and fewer requirements to the designer's help. Expand
Learning MDP Action Models Via Discrete Mixture Trees
TLDR
A regression tree algorithm in which each leaf node is modeled as a finite mixture of deterministic functions is introduced, which is approximated via a greedy set cover. Expand
Dynamic abstraction in reinforcement learning via clustering
TLDR
This work considers a graph theoretic approach for automatic construction of options in a dynamic environment and considers building a map that includes preliminary indication of the location of "interesting" regions of the state space, where the value gradient is significant and additional exploration might be beneficial. Expand
AUTOMATIC COMPLEXITY REDUCTION IN REINFORCEMENT LEARNING
TLDR
This work derives the knowledge of complexity reduction from partial solutions and provides algorithms for automated dimension reduction in RL and proposes the cascading decomposition algorithm based on the spectral analysis on a normalized graph Laplacian to decompose a problem into several subproblems and then conduct parameter relevance analysis on each subproblem to perform dynamic state abstraction. Expand
AUTOMATED DISCOVERY OF OPTIONS IN REINFORCEMENT LEARNING
  • 2003
AI planning benefits greatly from the use of temporally-extended or macroactions. Macro-actions allow for faster and more efficient planning as well as the reuse of knowledge from previous solutions.Expand
...
1
2
3
...