Richard Dearden

Learn More
Markov decision processes (MDPs) have proven to be popular models for decision-theoretic planning, but standard dynamic programming algorithms for solving MDPs rely on explicit, state-based specifications and computations. To alleviate the combinatorial problems associated with such methods, we propose new representational and computational techniques for(More)
A central problem in learning in complex environments is balancing exploration of untested actions against exploitation of actions that are known to be good. The benefit of exploration can be estimated using the classical notion of Value of Information—the expected improvement in future decision quality that might arise from the information acquired by(More)
Reinforcement learning systems are often concerned with balancing exploration of untested actions against exploitation of actions that are known to be good. The benefitof exploration can be estimated using the classical notion of Value of Information — the expected improvement in future decision quality arising from the information acquired by exploration.(More)
ion and Approximate Decision Theoretic Planning Richard Dearden and Craig Boutiliery Department of Computer Science University of British Columbia Vancouver, British Columbia CANADA, V6T 1Z4 email: dearden,cebly@cs.ubc.ca Abstract Markov decision processes (MDPs) have recently been proposed as useful conceptual models for understanding decision-theoretic(More)
There has been considerable work in AI on decisiontheoretic planning and planning under uncertainty. Unfortunately, all of this work suffers from one or more of the following limitations: 1) it relies on very simple models of actions and time, 2) it assumes that uncertainty is manifested in discrete action outcomes, and 3) it is only practical for very(More)
We describe an approach for exploiting structure in Markov Decision Processes with continuous state variables. At each step of the dynamic programming, the state space is dynamically partitioned into regions where the value function is the same throughout the region. We first describe the algorithm for piecewise constant representations. We then extend it(More)
Recently Markov decision processes and optimal control policies have been applied to the problem of decision-theoretic planning. However, the classical methods for generating optimal policies are highly intractable, requiring explicit enumeration of large state spaces. We explore a method for generating abstractions that allow approximately optimal policies(More)
This paper shows how state-of-the-art state estimation techniques can be used to provide efficient solutions to the difficult problem of real-time diagnosis in mobile robots. The power of the adopted estimation techniques resides in our ability to combine particle filters with classical algorithms, such as Kalman filters. We demonstrate these techniques in(More)
How seizures start is a major question in epilepsy research. Preictal EEG changes occur in both human patients and animal models, but their underlying mechanisms and relationship with seizure initiation remain unknown. Here we demonstrate the existence, in the hippocampal CA1 region, of a preictal state characterized by the progressive and global increase(More)