That and There: Judging the Intent of Pointing Actions with Robotic Arms

  title={That and There: Judging the Intent of Pointing Actions with Robotic Arms},
  author={Malihe Alikhani and Baber Khalid and Rahul Shome and Chaitanya Mitash and Kostas E. Bekris and Matthew Stone},
Collaborative robotics requires effective communication between a robot and a human partner. This work proposes a set of interpretive principles for how a robotic arm can use pointing actions to communicate task information to people by extending existing models from the related literature. These principles are evaluated through studies where English-speaking human subjects view animations of simulated robots instructing pick-and-place tasks. The evaluation distinguishes two classes of pointing… 

Figures and Tables from this paper

Task-driven Perception and Manipulation for Constrained Placement with No Shape Priors
A perception method is proposed to track and update the object representation during motion execution, which respects physical and geometric constraints, and is integrated with manipulation task planning architecture for a dual-arm manipulator to discover efficient solutions for the constrained placement task with minimal sensing.
Scalable, physics-aware 6D pose estimation for robot manipulation
Algorithms to generate and validate object poses online based on the objects’ geometry andbased on the physical consistency of their scene-level interactions are proposed, providing robustness even when there exists a domain gap between the synthetic training and the real test scenarios.
Robotic Arm Control System Based on AI Wearable Acceleration Sensor
The experimental results show that the average matching rate of the manipulator control system based on AI wearable acceleration sensor is as high as 88.89%, and the stability of the feature descriptor is high, and the research method is presented.
Chapter 5 Robotics as an Enabler of Resiliency to Disasters: Promises and Pitfalls
The maturity of robotics technology is discussed and the needed advances that will allow robots to become more capable and robust in disaster response measures are explored, which will help in making human and natural environments preemptively more resilient without compromising long-term prospects for economic development.


Legible robot pointing
A mathematical model is proposed that enables robots to generate pointing configurations that make the goal object as clear as possible - pointing configuration that are legible.
Robot Deictics: How Gesture and Context Shape Referential Communication
The results show that gestures which come into physical contact with the object offer the highest overall communicative accuracy and that specific settings benefit from the use of particular types of gestures.
Human-Robot Collaboration: From Psychology to Social Robotics
This survey reviews neuroscientific and psychological findings of the sensorimotor patterns that govern HHI and view them in a robotics context, and focuses on the mechanisms that are required for active, physical human-robot collaboration.
From embodied to socially embedded agents – Implications for interaction-aware robots
An Experimental Study for Identifying Features of Legible Manipulator Paths
The extensive experimental data provide insights into the features of motion that make certain paths more legible for humans than other paths.
Placing Objects in Gesture Space: Toward Incremental Interpretation of Multimodal Spatial Descriptions
This paper describes a real-time system to evaluate the separate and joint contributions of the modalities and shows that gestures not only help to improve the overall system performance, even if to a large extent they encode redundant information, but also result in earlier final correct interpretations.
Investigating models of social development using a humanoid robot
  • B. Scassellati
  • Psychology
    Proceedings of the International Joint Conference on Neural Networks, 2003.
  • 2003
A novel architecture called embodied theory of mind was developed to link high-level cognitive skills to the low-level perceptual abilities of a humanoid robot.
A Survey of Nonverbal Signaling Methods for Non-Humanoid Robots
This monograph presents a comprehensive review of literature related to the generation and usage of nonverbal signals that facilitate legibility of non-humanoid robot state and behavior and discusses issues that must be considered during nonverbal signaling and open research areas.
Deictic human/robot interaction
Grounding Verbs of Motion in Natural Language Commands to Robots
This work presents an algorithm for understanding natural language commands with three components, creating a cost function that scores the language according to how well it matches a candidate plan in the environment, defined as the log-likelihood of the plan given the command.