Bernhard Hengst

Learn More
An open problem in reinforcement learning is discovering hierarchical structure. HEXQ, an algorithm which automatically attempts to decompose and solve a model-free factored MDP hierarchically is described. By searching for aliased Markov sub-space regions based on the state variables the algorithm uses temporal and state abstraction to construct a(More)
Competing at the RoboCup 2000 Sony legged robot league, the UNSW team won both the challenge competition and all their soccer matches, emerging the outright winners for this league against eleven other international teams. The main advantage that the UNSW team had was speed. A major contributor to the speed was a novel omnidirectional locomotion method(More)
This thesis addresses the open problem of automatically discovering hierarchical structure in reinforcement learning. Current algorithms for reinforcement learning fail to scale as problems become more complex. Many complex environments empirically exhibit hierarchy and can be modelled as interrelated subsystems, each in turn with hierarchic structure.(More)
This paper introduces an optimised method for extracting natural landmarks to improve localisation during RoboCup soccer matches. The method uses modified 1D SURF features extracted from pixels on the robot’s horizon. Consistent with the original SURF algorithm, the extracted features are robust to lighting changes, scale changes, and small changes in(More)
One of the challenges of rescue robotics is to create robots that can autonomously traverse rough, unstructured terrain. Although mechanical engineering can produce very capable robots, mechanical engineering alone will not drive them. In this paper, we present a terrain feature extractor that can be taught to find significant features in range images of(More)
Search and rescue robots are intended to work as aids to human rescuers. One of the research challenges is how to combine autonomous operation with operator control. This paper presents a single software system which independently operates several physically distinct robots. One operator can coordinate several robots by switching their individual mode(More)
This paper describes the 2003 world champion legged robot soccer team, rUNSWift. The 2003 rUNSWift team is enhanced in a number of ways over previous teams; both long and short range collaboration between team members was carefully crafted, a new method of ball localization was used when the ball was close, new tools were developed for developing filters(More)
Hierarchical reinforcement learning methods have not been able to simultaneously abstract and reuse subtasks with discounted value functions. The contribution of this paper is to introduce two completion functions that jointly decompose the value function hierarchically to solve this problem. The significance of this result is that the benefits of(More)
A challenge in applying reinforcement learning to large problems is how to manage the explosive increase in storage and time complexity. This is especially problematic in multi-agent systems, where the state space grows exponentially in the number of agents. Function approximation based on simple supervised learning is unlikely to scale to complex domains(More)