#### Filter Results:

- Full text PDF available (8)

#### Publication Year

2004

2014

- This year (0)
- Last 5 years (2)
- Last 10 years (4)

#### Publication Type

#### Co-author

#### Journals and Conferences

#### Key Phrases

Learn More

- Norm Ferns, Prakash Panangaden, Doina Precup
- AAAI
- 2004

Markov decision processes (MDPs) offer a popular mathematical tool for planning and learning in the presence of uncertainty (Boutilier, Dean, & Hanks 1999). MDPs are a standard formalism for describing multi-stage decision making in probabilistic environments. The objective of the decision making is to maximize a cumulative measure of longterm performance,… (More)

- Norm Ferns, Prakash Panangaden, Doina Precup
- UAI
- 2005

We present metrics for measuring state similarity in Markov decision processes (MDPs) with infinitely many states, including MDPs with continuous state spaces. Such metrics provide a stable quantitative analogue of the notion of bisimulation for MDPs, and are suitable for use in MDP approximation. We show that the optimal value function associated with a… (More)

A popular approach to solving large probabilistic systems relies on aggregating states based on a measure of similarity. Many approaches in the literature are heuristic. A number of recent methods rely instead on metrics based on the notion of bisimulation, or behavioral equivalence between states (Givan et al., 2003; Ferns et al., 2004). An integral… (More)

- Norm Ferns, Prakash Panangaden, Doina Precup
- SIAM J. Comput.
- 2011

In recent years, various metrics have been developed for measuring the behavioural similarity of states in probabilistic transition systems [Desharnais et al., Proceedings of CONCUR, (1999), pp. 258-273, van Breugel and Worrell, Proceedings of ICALP, (2001), pp. 421-432]. In the context of finite Markov decision processes, we have built on these metrics to… (More)

Approximation techniques for labelled Markov processes on continuous state spaces were developed by Desharnais, Gupta, Jagadeesan and Panangaden. However, it has not been clear whether this scheme could be used in practice since it involves inverting a stochastic kernel. We describe a Monte-Carlobased implementation scheme for this approximation algorithm.… (More)

- Norm Ferns, Doina Precup, Sophia Knight
- Horizons of the Mind
- 2014

We transfer a notion of quantitative bisimilarity for labelled Markov processes [1] to Markov decision processes with continuous state spaces. This notion takes the form of a pseudometric on the system states, cast in terms of the equivalence of a family of functional expressions evaluated on those states and interpreted as a real-valued modal logic. Our… (More)

- Stewart Heitmann, Norm Ferns, Michael Breakspear
- Front. Neurorobot.
- 2011

Computational models of neuromotor control require forward models of limb movement that can replicate the natural relationships between muscle activation and joint dynamics without the burdens of excessive anatomical detail. We present a model of a three-link biomechanical limb that emphasizes the dynamics of limb movement within a simplified… (More)

- Norm Ferns, Doina Precup
- UAI
- 2014

Bisimulation is a notion of behavioural equivalence on the states of a transition system. Its definition has been extended to Markov decision processes, where it can be used to aggregate states. A bisimulation metric is a quantitative analog of bisimulation that measures how similar states are from a the perspective of long-term behavior. Bisimulation… (More)

- ‹
- 1
- ›