Teaching System for Multimodal Object Categorization by Human-Robot Interaction in Mixed Reality

  title={Teaching System for Multimodal Object Categorization by Human-Robot Interaction in Mixed Reality},
  author={Lotfi El Hafi and Hitoshi Nakamura and Akira Taniguchi and Yoshinobu Hagiwara and Tadahiro Taniguchi},
  journal={2021 IEEE/SICE International Symposium on System Integration (SII)},
As service robots are becoming essential to support aging societies, teaching them how to perform general service tasks is still a major challenge preventing their deployment in daily-life environments. In addition, developing an artificial intelligence for general service tasks requires bottom-up, unsupervised approaches to let the robots learn from their own observations and interactions with the users. However, compared to the top-down, supervised approaches such as deep learning where the… 

Figures and Tables from this paper

Augmented Reality and Robotics: A Survey and Taxonomy for AR-enhanced Human-Robot Interaction and Robotic Interfaces
This paper contributes to a taxonomy of augmented reality and robotics based on a survey of 460 research papers and formulate key challenges and opportunities to guide and inform future research in AR and robotics.
Semiotically adaptive cognition: toward the realization of remotely-operated service robots for the new normal symbiotic society
This paper argues that the development of semiotically adaptive cognitive systems is key to the installation of service robotics technologies in the authors' service environments and describes three challenges: the learning of local knowledge, the acceleration of onsite and online learning, and the augmentation of human–robot interactions.
Bootstrapping Concept Formation in Small Neural Networks
It is hypothesized that the neuronal pools representing relational information can be considered as primordial Concepts, which may in a similar way be present in some pre-linguistic animals, too.


System for augmented human–robot interaction through mixed reality and robot training by non-experts in customer service environments
A complete robot system that has been given the highest evaluation score at the Customer Interaction Task of the Future Convenience Store Challenge at the World Robot Summit 2018 is presented, which implements several key technologies including a hierarchical spatial concepts formation for general robot task planning and a mixed reality interface to enable users to intuitively visualize the current state of the robot perception and naturally interact with it.
Interactive Robot Knowledge Patching Using Augmented Reality
A novel Augmented Reality approach to address the challenging problems of diagnosing, teaching, and patching interpretable knowledge of a robot through a Temporal And-Or graph of opening bottles, which yields a hierarchical structure that captures the compositional nature of the given task.
Long-term learning of concept and word by robots: Interactive learning framework and preliminary results
This paper develops a practical on-line learning algorithm that solves three remaining problems in the previous study and proposes an interactive learning framework, in which the proposed on- line learning algorithm is embedded.
Grounding of word meanings in multimodal concepts using LDA
This paper proposes LDA-based framework for multimodal categorization and words grounding for robots and provides a relevance measure that encodes the degree of connection between words and modalities.
New robot technology challenge for convenience store
  • K. Wada
  • Computer Science
    2017 IEEE/SICE International Symposium on System Integration (SII)
  • 2017
The design award and overview of robot technology challenge are explained, in order to create new image of future convenience store with robot technology and ideas about new services and convenience store design are asked the public.
ROS: an open-source Robot Operating System
This paper discusses how ROS relates to existing robot software frameworks, and briefly overview some of the available application software which uses ROS.
A Rack-scale Optical disc library System, or ROS in short, which provides a PB-level total capacity and inline accessibility on thousands of optical discs built within a 42U Rack and is able to effectively hide and virtualize internal complex operational behaviors and be easily deployable in datacenters.
Fast Explicit Diffusion for Accelerated Features in Nonlinear Scale Spaces
A novel and fast multiscale feature detection and description approach that exploits the benefits of nonlinear scale spaces and introduces a Modified-Local Difference Binary (M-LDB) descriptor that is highly efficient, exploits gradient information from the non linear scale space, is scale and rotation invariant and has low storage requirements.
Network video monitoring system based on OpenCV (open source computer vision library)
The utility model relates to a video monitoring system based on an OpenCV (open source computer vision library) that can receive information transmitted by the video camera through a Camera Link interface, store data in the memory unit after decoding and send the data to the remote client computer through an ARM bus interface.
  • M. Arditti
  • Medicine
    Journal of the Irish Medical Association
  • 1957
This chapter discusses Garona, Thrall, Jaina Proudmoore, Sergra Darkhorn, Naz'grel, Ariana Trueshot, Logrosh Trueshots, and more.