# Robot introspection through learned hidden Markov models

@article{Fox2006RobotIT, title={Robot introspection through learned hidden Markov models}, author={M. Fox and M. Ghallab and G. Infantes and D. Long}, journal={Artif. Intell.}, year={2006}, volume={170}, pages={59-113} }

In this paper we describe a machine learning approach for acquiring a model of a robot behaviour from raw sensor data. We are interested in automating the acquisition of behavioural models to provide a robot with an introspective capability. We assume that the behaviour of a robot in achieving a task can be modelled as a finite stochastic state transition system. Beginning with data recorded by a robot in the execution of a task, we use unsupervised learning techniques to estimate a hidden… Expand

#### Figures and Topics from this paper

#### 73 Citations

Analysis of multimodal Bayesian nonparametric autoregressive hidden Markov models for process monitoring in robotic contact tasks

- Computer Science
- 2019

A nonparametric prior is used that endows the hidden Markov models with an unbounded number of hidden states for a given robot skill (or subtask) and is compared with other methods for skill and anomaly identification. Expand

Learning Behaviors Models for Robot Execution Control

- Computer Science
- ICAPS
- 2006

This work proposes to learn behaviors models as structured stochastic processes: Dynamic Bayesian Network and believes that the proposed approach remains generic and can be used to learn complex behaviors other than navigation and for other autonomous systems. Expand

Understanding human intentions via Hidden Markov Models in autonomous mobile robots

- Computer Science
- 2008 3rd ACM/IEEE International Conference on Human-Robot Interaction (HRI)
- 2008

This paper proposes an approach that allows a robot to detect intentions of others based on experience acquired through its own sensory-motor capabilities, then using this experience while taking the perspective of the agent whose intent should be recognized. Expand

Learning the behavior model of a robot

- Computer Science
- Auton. Robots
- 2011

A general framework for learning from observation data the behavior model of a robot when performing a given task is proposed and it is shown how such a probabilistic model can be learned and used to improve, on line, the robot behavior with respect to a specific environment and user preferences. Expand

Maximum likelihood estimation of sensor and action model functions on a mobile robot

- Engineering, Computer Science
- 2008 IEEE International Conference on Robotics and Automation
- 2008

This paper presents an adaptation of the Expectation-Maximization (EM) algorithm to enable a mobile robot to learn both its action and sensor model functions, starting without an accurate version of either. Expand

Robotic Behavior Prediction Using Hidden Markov Models

- Computer Science
- ArXiv
- 2014

This paper presents an approach to modeling behaviors of dynamic agents in order to empower robots with the ability to predict the agent's actions and identify the behavior the agent is executing in real time. Expand

Using learned action models in execution monitoring

- Engineering
- 2006

Planners reason with abstracted models of the behaviours they use to construct plans. When plans are turned into the instructions that drive an executive, the real behaviours interacting with the… Expand

A Latent State-Based Multimodal Execution Monitor with Anomaly Detection and Classification for Robot Introspection

- Engineering
- 2019

Robot introspection is expected to greatly aid longer-term autonomy of autonomous manipulation systems. By equipping robots with abilities that allow them to assess the quality of their sensory data,… Expand

Nonparametric Bayesian Modeling of Multimodal Time Series

- Computer Science
- 2020

In this chapter, we take a Bayesian nonparametric approach in defining a prior on the hidden Markov model that allows for flexibility in addressing the problem of modeling the complex dynamics during… Expand

Integration of programming and learning in a control language for autonomous robots performing everyday activities

- Computer Science
- 2008

In this thesis a robot control language is introduced, which allows to describe declaratively and execute complete learning processes in the program, which includes the identification and recording of experiences, the learning process itself, and the integration of the learning result into the program. Expand

#### References

SHOWING 1-10 OF 61 REFERENCES

Inferring finite automata with stochastic output functions and an application to map learning

- Computer Science
- 26th Annual Symposium on Foundations of Computer Science (sfcs 1985)
- 1985

An exploration algorithm is provided to learn the correct structure of a reduced deterministic finite automaton with probability 1−δ, given as inputs δ, an upper boundm on the number of states, a distinguishing sequences, and a lower bound α>1/2 on the probability of observing the correct output at any state. Expand

Unsupervised learning of probabilistic models for robot navigation

- Computer Science
- Proceedings of IEEE International Conference on Robotics and Automation
- 1996

An algorithm is described that adjusts the probabilities of the initial Markov model by passively observing the robot's interactions with its environment and learns good Markov models with a small amount of training data. Expand

A Method for Clustering the Experiences of a Mobile Robot that Accords with Human Judgments

- Computer Science
- AAAI/IAAI
- 2000

This work presents an unsupervised learning method that allows a robotic agent to identify and represent qualitatively different outcomes of actions and shows that the models acquired by the robot correlate surprisingly well with human models of the environment. Expand

Coping with Uncertainty in Map Learning

- Mathematics, Computer Science
- Machine Learning
- 2004

A framework for describing map-learning problems in which the measurements taken by the robot are subject to known errors is developed, and approaches to learning maps under such conditions based on Valiant's probably approximately correct learning model are investigated. Expand

Policy Recognition in the Abstract Hidden Markov Model

- Computer Science
- J. Artif. Intell. Res.
- 2002

This paper introduces the Abstract Hidden Markov Model (AHMM), a novel type of stochastic processes, provide its dynamic Bayesian network (DBN) structure and analyse the properties of this network, and proposes a novel plan recognition framework based on the AHMM as the plan execution model. Expand

Passive Distance Learning for Robot Navigation

- Computer Science
- ICML
- 1996

GROW-BW is presented, an unsupervised and passive distance learning algorithm that overcomes the problem that the robot can never be sure about its location if it is not allowed to reduce its uncertainty by asking a teacher or executing localization actions. Expand

Learning Geometrically-Constrained Hidden Markov Models for Robot Navigation: Bridging the Topological-Geometrical Gap

- Computer Science
- J. Artif. Intell. Res.
- 2002

A formal framework for incorporating readily available odometric information and geometrical constraints into both the models and the algorithm that learns them is described, which can help to generate better solutions and require fewer iterations, while being robust in the face of data reduction. Expand

Hidden Markov Model} Induction by Bayesian Model Merging

- Mathematics, Computer Science
- NIPS
- 1992

The algorithm is compared with the Baum-Welch method of estimating fixed-size models, and it is found that it can induce minimal HMMs from data in cases where fixed estimation does not converge or requires redundant parameters to converge. Expand

Parametric Hidden Markov Models for Gesture Recognition

- Computer Science
- IEEE Trans. Pattern Anal. Mach. Intell.
- 1999

The approach is to extend the standard hidden Markov model method of gesture recognition by including a global parametric variation in the output probabilities of the HMM states by forming an expectation-maximization (EM) method for training the parametric HMM. Expand

An Architecture for Autonomy

- Computer Science, Engineering
- Int. J. Robotics Res.
- 1998

An integrated architecture that allows a mobile robot to plan its tasks—taking into account temporal and domain constraints, to perform corresponding actions and to con trol their execution in real-time—while being reactive to possible events is described. Expand