Augmented reality and haptic interfaces for robot‐assisted surgery

  title={Augmented reality and haptic interfaces for robot‐assisted surgery},
  author={Tomonori Yamamoto and Niki Abolhassani and Sung Jung and Allison M. Okamura and Timothy N. Judkins},
  journal={The International Journal of Medical Robotics and Computer Assisted Surgery},
Current teleoperated robot‐assisted minimally invasive surgical systems do not take full advantage of the potential performance enhancements offered by various forms of haptic feedback to the surgeon. Direct and graphical haptic feedback systems can be integrated with vision and robot control systems in order to provide haptic feedback to improve safety and tissue mechanical property identification. 

13 Haptic Feedback Modalities for Minimally Invasive Surgery

This chapter introduces haptic feedback modalities for robot-assisted minimally invasive surgical platforms, such as STIFF-FLOP.

Online model estimation and haptic characterization for robotic-assisted minimally invasive surgery

Online soft tissue characterization is important for robotic-assisted minimally invasive surgery (RAMIS) to achieve a precise and stable robotic control with haptic feedback. The traditional linear

A review of haptic feedback in tele-operated robotic surgery.

During traditional surgery, the surgeons’ hands are in direct contact with organs, and surgeons rely on the sense of touch to perform surgery. In teleoperated robotic systems, all physical connecti...

Incremental potential field based haptic guidance for medical simulation

A user is physically guided through the ideal motion path by a haptic interfacing device, giving the subject a kinesthetic understanding of the task, and this is used in the simulation of tracheal reconstruction surgery to investigate the feasibility of deploying newtracheal implants.

Shared control of a medical robot with haptic guidance

The experiment results demonstrate that comparing with vision only guidance, the shared control with vision and haptics improved the accuracy and efficiency of surgical robotic manipulation, where the tool-position error distance and execution time are reduced.

Surgical Instruments Motion Safety Constraint Based on Haptic Virtual Fixture

Experiments show that the three haptic virtual fixtures designed are simple and effective, and can play a safe and protective role.

A review of haptic feedback in tele-operated robotic surgery

It is hard to imagine how to operate with no haptic sensations, and it is surprising that commercially available robots didn’t include until now any Haptic Feedback, despite reports about tissue injury, and inability to perform complex manipulation.

Applications of Haptics in Medicine

This chapter provides an overview of the various available technologies to perform haptic feedback and details on how haptic guidance can enhance surgical skill acquisition and the advantages of usual supervised hands-on training and the ones offered by computer-based training: dual-user training simulators.

Recent Challenges for Haptic Interface and Control for Robotic Assisted Surgical Training System: A Review

In depth review and highlights of the major challenging factors for haptic control interfacing, which are required to overcome in the future development of intelligent and highly integrated surgical training system for robotic surgery.

A tactile sensing and feedback system for tumor localization

  • Min LiShan LuoGuanghua Xu
  • Medicine
    2016 13th International Conference on Ubiquitous Robots and Ambient Intelligence (URAI)
  • 2016
A tactile sensing and feedback system, consisting of a colored tactile information display and pneumatic multi-fingered haptic feedback, that is validated by palpating two phantom organs to locate tissue tumors is described.



Haptic feedback in robot-assisted minimally invasive surgery

The designs of existing commercial RMIS systems are not conducive for force feedback, and creative solutions are needed to create compelling tactile feedback systems.

Effects of haptic and graphical force feedback on teleoperated palpation

Results show that direct haptic force feedback minimizes applied forces to the tissue, while coupled haptic and graphical force Feedback minimizes subject task error and for experienced surgeons, hapticforce feedback substantially reduced task error independent of graphical feedback.

Effect of sensory substitution on suture-manipulation forces for robotic surgical systems.

Virtual fixtures: Perceptual tools for telerobotic manipulation

  • L. Rosenberg
  • Business
    Proceedings of IEEE Virtual Reality Annual International Symposium
  • 1993
The beneficial effects of corrupting the link between operator and remote environment by introducing abstract perceptual information into the interface called virtual fixtures are described.

Tissue property estimation and graphical display for teleoperated robot-assisted surgery

An automated tissue property estimation method and a real-time graphical overlay that allow an operator to discriminate hard and soft tissues are presented.

Vision-assisted control for manipulation using virtual fixtures

The design and implementation of a vision-based system for cooperative manipulation at millimeter to micrometer scales based on an admittance control algorithm that implements a broad class of guidance modes called virtual fixtures is presented.

Stability of Haptic Rendering: Discretization, Quantization, Time Delay, and Coulomb Effects

A study of this problem, which relates the maximum achievable object stiffness to the elements of the control loop, examines how the sampling rate, quantization, computational delay, and amplifier dynamics interact with the inertia, natural viscous, and Coulomb damping of the haptic device.

Robot-assisted Tactile Sensing for Minimally Invasive Tumor Localization

The results show that using the TSI under robotic control realizes an average 35% decrease in the maximum forces applied and a 50% increase in tumor detection accuracy when compared to manual manipulation of the same instrument, demonstrating that the detection of tumors using tactile sensing is highly dependent on how consistently the forces on the tactile sensing area are applied.

Vision-Based Assistance for Ophthalmic Micro-Surgery

Development and preliminary testing of a system for 6-DOF human-machine cooperative motion using vision-based virtual fixtures for applications in retinal micro-surgery and the effectiveness of the system in guiding a user along the surface and performing different sub-tasks within the resolution of the visual system is shown.