Hand gesture guided robot-assisted surgery based on a direct augmented reality interface


Radiofrequency (RF) ablation is a good alternative to hepatic resection for treatment of liver tumors. However, accurate needle insertion requires precise hand-eye coordination and is also affected by the difficulty of RF needle navigation. This paper proposes a cooperative surgical robot system, guided by hand gestures and supported by an augmented reality (AR)-based surgical field, for robot-assisted percutaneous treatment. It establishes a robot-assisted natural AR guidance mechanism that incorporates the advantages of the following three aspects: AR visual guidance information, surgeon's experiences and accuracy of robotic surgery. A projector-based AR environment is directly overlaid on a patient to display preoperative and intraoperative information, while a mobile surgical robot system implements specified RF needle insertion plans. Natural hand gestures are used as an intuitive and robust method to interact with both the AR system and surgical robot. The proposed system was evaluated on a mannequin model. Experimental results demonstrated that hand gesture guidance was able to effectively guide the surgical robot, and the robot-assisted implementation was found to improve the accuracy of needle insertion. This human-robot cooperative mechanism is a promising approach for precise transcutaneous ablation therapy.

DOI: 10.1016/j.cmpb.2013.12.018

Extracted Key Phrases

13 Figures and Tables

Showing 1-10 of 24 references

A robotic system for overlapping radiofrequency ablation in large tumor treatment

  • L Yang, R Wen, J Qin, C K Chui, K B Lim, S K Y Chang
  • 2010
Highly Influential
3 Excerpts

Gesture Based Surgical Manipulation of a da Vinci Robot Using a Kinect

  • 2013
1 Excerpt

Does a robotic scrub nurse improve economy of movements? in

  • J P Wachs, M Jacob, Y T Li, G Akingba
  • 2012
1 Excerpt

Robot-assisted RF ablation with interactive planning and mixed reality guidance

  • R Wen, C B Chng, C K Chui, K B Lim, S H Ong, S K Y Chang
  • 2012
1 Excerpt
Showing 1-10 of 10 extracted citations