• Publications
  • Influence
Equivalence notions and model minimization in Markov decision processes
tl;dr
We present a non-trivial notion of state equivalence that is based upon the notion of bisimulation from the literature on concurrent processes that guarantees the optimal policy for the reduced model immediately induces a corresponding Optimal Policy for the original model. Expand
  • 264
  • 37
FF-Replan: A Baseline for Probabilistic Planning
tl;dr
This paper gives the first technical description of FF-Replan and provides an analysis of its results on all of the recent IPPC-04 andIPPC-06 domains. Expand
  • 264
  • 33
Probabilistic Planning via Determinization in Hindsight
tl;dr
This paper investigates hindsight optimization as an approach for leveraging the significant advances in deterministic planning for action selection in probabilistic domains. Expand
  • 127
  • 16
Bounded Parameter Markov Decision Processes
tl;dr
We introduce the notion of a bounded parameter Markov decision process (BMDP) as a generalization of the familiar exact MDP, which can be used to represent variation or uncertainty concerning the parameters of sequential decision problems in cases where no prior probabilities on the parameter values are available. Expand
  • 118
  • 16
Bounded-parameter Markov decision processes
tl;dr
In this paper, we introduce the notion of a {\em bounded parameter Markov decision process\/} as a generalization of the traditional {\em exact\/} MDP. Expand
  • 165
  • 14
Approximate Policy Iteration with a Policy Language Bias
tl;dr
We explore approximate policy iteration, replacing the usual cost-function learning step with a learning step in policy space. Expand
  • 198
  • 12
Model Minimization in Markov Decision Processes
tl;dr
We use the notion of stochastic bisimulation homogeneity to analyze planning problems represented as Markov decision processes (MDPs). Expand
  • 206
  • 8
Model Reduction Techniques for Computing Approximately Optimal Solutions for Markov Decision Processes
tl;dr
We present a method for solving implicit (factored) Markov decision processes (MDPs) with very large state spaces using a factored representation of an MDP and an e-homogeneous partition of the state space. Expand
  • 113
  • 7
Landmark Extraction via Planning Graph Propagation
The planner GRAPHPLAN is based on an efficient propagation of reachability information which then effectively gu ides a search for a valid plan. We propose a framework in which a broader class ofExpand
  • 54
  • 7
Inductive Policy Selection for First-Order MDPs
tl;dr
We select policies for large Markov Decision Processes (MDPs) with compact first-order representations that generalize well as the number of objects in the domain grows, potentially without bound. Expand
  • 88
  • 6