Martin Stolle

Learn More
Temporally extended actions (e.g., macro actions) have proven very useful in speeding up learning, ensuring robustness and building prior knowledge into AI systems. The options framework (Precup, 2000; Sutton, Precup & Singh, 1999) provides a natural way of incorporating such actions into reinforcement learning systems, but leaves open the issue of how good(More)
Libraries of trajectories are a promising way of creating policies for difficult problems. However, often it is not desirable or even possible to create a new library for every task. We present a method for transferring libraries across tasks, which allows us to build libraries by learning from demonstration on one task and apply them to similar tasks.(More)
We present a novel approach to legged locomotion over rough terrain that is thoroughly rooted in optimization. This approach relies on a hierarchy of fast, anytime algorithms to plan a set of footholds, along with the dynamic body motions required to execute them. Components within the planning framework coordinate to exchange plans, cost-to-go estimates,(More)
We present several algorithms that aim to advance the state-of-the-art in reinforcement learning and planning algorithms. One key idea is to transfer knowledge across problems by representing it using local features. This idea is used to speed up a dynamic programming based generalized policy iteration. We then present a control approach that uses a library(More)
We present a control approach that uses a library of trajectories to establish a global control law or policy. This is an alternative to methods for finding global policies based on value functions using dynamic programming and also to using plans based on a single desired trajectory. Our method has the advantage of providing reasonable policies much faster(More)
AI planning benefits greatly from the use of temporally-extended or macroactions. Macro-actions allow for faster and more efficient planning as well as the reuse of knowledge from previous solutions. In recent years, a significant amount of research has been devoted to incorporating macro-actions in learned controllers, particularly in the context of(More)
This work focuses on the remote sensing of thermal updrafts under cumulus clouds for glider UAVs. Previous remote estimation techniques assume unlimited thermal lifespans under the clouds. Naturally occuring thermals are however of limited duration which increases the risk of an outlanding when utilizing these methods. By observing the evolution of the(More)
Autonomous soaring is a promising approach to augment the endurance of small UAVs. Most of the existing work on this field relies on accelerometers and/or GPS receivers to sense thermals in the proximity of the vehicle. However, thermal updrafts are often visually indicated by cumulus clouds that are well characterized by their sharp baselines. This paper(More)