• Corpus ID: 15551054

Safety in AI-HRI: Challenges Complementing User Experience Quality

  title={Safety in AI-HRI: Challenges Complementing User Experience Quality},
  author={Richard Gabriel Freedman and Shlomo Zilberstein},
  booktitle={AAAI Fall Symposia},
Contemporary research in human-robot interaction (HRI) predominantly focuses on the user’s experience while controlling a robot. However, with the increased deployment of artificial intelligence (AI) techniques, robots are quickly becoming more autonomous in both academic and industrial experimental settings. In addition to improving the user’s interactive experience with AI-operated robots through personalization, dialogue, emotions, and dynamic behavior, there is also a growing need to… 

Figures from this paper

A novel multi-step reinforcement learning method for solving reward hacking
A new multi-step state-action value algorithm is proposed to solve the problem of reward hacking by using a new return function, which alters the discount of future rewards and no longer stresses the immediate reward as the main influence when selecting the current state action.
Shielded Decision-Making in MDPs
This work presents the concept of a shield that forces decision-making to provably adhere to safety requirements with high probability, and presents a method to compute probabilities of decision making regarding temporal logic constraints.
Safe Reinforcement Learning Using Probabilistic Shields
The concept of a probabilistic shield that enables RL decision-making to adhere to safety constraints with high probability is introduced and used to realize a shield that restricts the agent from taking unsafe actions, while optimizing the performance objective.
Safe Reinforcement Learning via Probabilistic Shields
This paper introduces the concept of a probabilistic shield that enables decision-making to adhere to safety constraints with high probability and discusses tradeoffs between sufficient progress in exploration of the environment and ensuring safety.
Verification of Uncertain POMDPs Using Barrier Certificates
This work casts the POMDP problem into a switched system scenario, takes advantage of this switched system characterization and proposes a method based on barrier certificates for optimality and/or safety verification, and shows that the verification task can be carried out computationally by sum-of-squares programming.
Robustness Verification for Classifier Ensembles
A formal verification procedure that decides whether a classifier ensemble is robust against arbitrary randomized attacks, using SMT and MILP encodings to compute optimal randomized attacks or proving that there is no attack inducing a certain expected loss, is given.


Human expectations of social robots
It is concluded that increasing social capabilities in robots can produce an expectations gap where humans develop unrealistically high expectations of social robots due to generalization from human mental models, which could ironically result in less effective collaborations as robot capabilities improve.
Experiences developing socially acceptable interactions for a robotic trash barrel
Interactions with a trash barrel robot are developed and tested to better understand the implicit protocols for public interaction, showing that people most welcome the robot's presence when they need its services and it actively advertises its intent through movement.
Designing robot learners that ask good questions
  • M. Cakmak, A. Thomaz
  • Computer Science
    2012 7th ACM/IEEE International Conference on Human-Robot Interaction (HRI)
  • 2012
This paper identifies three types of questions (label, demonstration and feature queries) and discusses how a robot can use these while learning new skills and provides guidelines for designing question asking behaviors on a robot learner.
Integrating human observer inferences into robot motion planning
This work introduces the notion of an observer into motion planning, and formalizes predictability and legibility as properties of motion that naturally arise from the inferences in opposing directions that the observer makes, drawing on action interpretation theory in psychology.
The role of roles: Physical cooperation between humans and robots
A formal analysis of human–robot cooperative load transport is presented and results show tradeoffs between subjective and objective performance measures stating a clear objective advantage of the proposed dynamic role allocation scheme.
Learning interaction for collaborative tasks with probabilistic movement primitives
This paper introduces the use of Probabilistic Movement Primitives (ProMPs) to devise an interaction method that both recognizes the action of a human and generates the appropriate movement primitive of the robot assistant.
Toward safe close-proximity human-robot interaction with standard industrial robots
This work presents a real-time safety system capable of allowing safe human-robot interaction at very low distances of separation, without the need for robot hardware modification or replacement, resulting in robust real- time performance.
Concrete Problems in AI Safety
A list of five practical research problems related to accident risk, categorized according to whether the problem originates from having the wrong objective function, an objective function that is too expensive to evaluate frequently, or undesirable behavior during the learning process, are presented.
Decision-making authority, team efficiency and human worker satisfaction in mixed human–robot teams
It is found that an autonomous robot can outperform a human worker in the execution of part or all of the process of task allocation, and that people preferred to cede their control authority to the robot than to human teammates only.