Automating Surgical Peg Transfer: Calibration With Deep Learning Can Exceed Speed, Accuracy, and Consistency of Humans

  title={Automating Surgical Peg Transfer: Calibration With Deep Learning Can Exceed Speed, Accuracy, and Consistency of Humans},
  author={Minho Hwang and Jeffrey Ichnowski and Brijen Thananjeyan and Daniel Seita and Samuel Paradis and Danyal Fer and Thomas Low and Ken Goldberg},
  journal={IEEE Transactions on Automation Science and Engineering},
— Peg transfer is a well-known surgical training task in the Fundamentals of Laparoscopic Surgery (FLS). While human surgeons teleoperate robots such as the da Vinci to perform this task with high speed and accuracy, it is challenging to automate. This paper presents a novel system and control method using a da Vinci Research Kit (dVRK) surgical robot and a Zivid depth sen-sor, and a human subjects study comparing performance on three variants of the peg-transfer task: unilateral, bilateral… 
1 Citations

Autonomous Peg Transfer—a Gateway to Surgery 4.0

  • T. D. NagyT. Haidegger
  • Computer Science
    2022 IEEE 10th Jubilee International Conference on Computational Cybernetics and Cyber-Medical Systems (ICCC)
  • 2022
An overview of the current state of the research in this particular area is presented, emphasizing benchmarks and different variations of the peg transfer training exercise.



Applying Depth-Sensing to Automated Surgical Manipulation with a da Vinci Robot

Results over 236 and 49 total block transfer attempts for the single- and bilateral-arm peg transfer cases suggest that reliability can be attained with 86.9% and 78.0% for each individual block, with respective block transfer speeds of 10.02 and 5.72 seconds.

Fast and Reliable Autonomous Surgical Debridement with Cable-Driven Robots Using a Two-Phase Calibration Procedure

A novel two-phase coarse-to-fine calibration method for debridement of raisins and pumpkin seeds as fragment phantoms and finds that without calibration, position errors average 4.55mm and the combination of Phase I and Phase II can reduce average error to 1.08mm.

Using Contact Forces and Robot Arm Accelerations to Automatically Rate Surgeon Skill at Peg Transfer

An automatic skill evaluation system that analyzes only the contact force with the task materials, the broad-bandwidth accelerations of the robotic instruments and camera, and the task completion time to reliably rated a surgeon's skill at robotic peg transfer via regression using features gathered from force, acceleration, and time sensors external to the robot.

Transferring Know-How for an Autonomous Camera Robotic Assistant

The results demonstrate that the use of learning from demonstration is a suitable method to perform autonomous camera guidance in collaborative surgical robotic platforms.

Intermittent Visual Servoing: Efficiently Learning Policies Robust to Instrument Changes for High-precision Surgical Manipulation

Results on a da Vinci Research Kit suggest that combining the coarse policy with half a second of corrections from the learned policy during each high-precision segment improves the success rate on the Fundamentals of Laparoscopic Surgery peg transfer task.

Learning by observation for surgical subtasks: Multilateral cutting of 3D viscoelastic and 2D Orthotropic Tissue Phantoms

Using the da Vinci Research Kit (DVRK) robotic surgical assistant, this work explores a “Learning By Observation” (LBO) approach where the robot identifies, segment, and parameterize motion sequences and sensor conditions to build a finite state machine (FSM) for each subtask.

Efficiently Calibrating Cable-Driven Surgical Robots With RGBD Fiducial Sensing and Recurrent Neural Networks

A novel approach to efficiently calibrate robotic surgical assistants by placing a 3D printed fiducial coordinate frames on the arm and end-effector that is tracked using RGBD sensing and considering 13 approaches to modeling to measure the coupling and history-dependent effects between joints.

Automating multi-throw multilateral surgical suturing with a mechanical needle guide and sequential convex optimization

Initial results suggest that the dVRK can perform suturing at 30% of human speed while completing 86% suture throws attempted, and the Suture Needle Angular Positioner (SNAP) results in a 3x error reduction in the needle pose estimate in comparison with the standard actuator.

Do laparoscopic skills transfer to robotic surgery?

A case study of trajectory transfer through non-rigid registration for a simplified suturing scenario

The applicability of the trajectory transfer algorithm proposed in [12] to the automation of suturing is studied and the success rate for learning from a single demonstration is high for moderate perturbation from the demonstration's initial conditions, and it gradually decreases for larger perturbations.