A System for Robotic Heart Surgery that Learns to Tie Knots Using Recurrent Neural Networks
@article{Mayer2006ASF, title={A System for Robotic Heart Surgery that Learns to Tie Knots Using Recurrent Neural Networks}, author={Hermann Georg Mayer and Faustino J. Gomez and Daan Wierstra and Istv{\'a}n Nagy and Alois Knoll and J{\"u}rgen Schmidhuber}, journal={2006 IEEE/RSJ International Conference on Intelligent Robots and Systems}, year={2006}, pages={543-548} }
Tying suture knots is a time-consuming task performed frequently during minimally invasive surgery (MIS). Automating this task could greatly reduce total surgery time for patients. Current solutions to this problem replay manually programmed trajectories, but a more general and robust approach is to use supervised machine learning to smooth surgeon-given training trajectories and generalize from them. Since knottying generally requires a controller with internal memory to distinguish between…Â
115 Citations
A System for Robotic Heart Surgery that Learns to Tie Knots Using Recurrent Neural Networks
- Computer Science
- 2006
Results obtained using long short-term memory RNNs trained by the recent Evolino algorithm show that this approach can significantly increase the efficiency of suture knot tying in MIS over preprogrammed control.
A Novel Method for Knot-Tying in Autonomous Robotic assisted Surgery Using Deep Learning
- Computer Science
- 2021
A novel method using deep learning based on Variational Autoencoders for robotic assistants to learn knot-tying with the Data Set of manual operation, instead of learning from video demonstrations is proposed.
Dynamic trajectory planning for robotic knot tying
- Medicine2016 IEEE International Conference on Real-time Computing and Robotics (RCAR)
- 2016
A new knot-tying method is proposed to enhance the quality of robotic Knot tying practice with low supervision and results confirm that suture loops can be successfully winded around the instrument without suture slippage.
A Preliminary Investigation Into the Feasibility of Semi-Autonomous Surgical Path Planning for a Mastoidectomy Using LSTM-Recurrent Neural Networks
- Medicine
- 2021
Almost captured surgical trajectories from trained surgeons were used to train a set of artificial neural networks (ANNs) using recorded surgeons' motions when manipulating a surgical instrument during procedure training using a surgery simulator, and it was found that even when trained on a small number of datasets, the ANNs converged and could generate output trajectories that were still assessed to be safe even when slight changes in the fiducial marker placement locations were given.
Human-Machine Collaborative surgery using learned models
- Medicine, Computer Science2011 IEEE International Conference on Robotics and Automation
- 2011
A novel surgical Human-Machine Collaborative system in which portions of a surgical task are performed autonomously under complete surgeon's control, and other portions manually, provides a safe and acceptable solution for surgical performance enhancement.
Surgical Automation for Multilateral Multi-Throw Suturing
- Medicine
- 2016
A novel mechanical needle guide and a framework for optimizing needle size, trajectory, and control parameters using sequential convex programming are presented and initial results suggest that the dVRK can perform suturing at 30% of human speed while completing 86% suture throws attempted.
Trajectory Planning of Knot-Tying Manipulation in Surgery
- Medicine
- 2012
To improve the surgical knot quality and accelerate healing for patients in surgery, trajectory planning of suture clamping point in knot-tying manipulation using a master-slave robot is discussed based on the surgeon’s manual motion of knot-TYing.
A CNN-based prototype method of unstructured surgical state perception and navigation for an endovascular surgery robot
- EngineeringMedical & Biological Engineering & Computing
- 2019
A novel convolutional neural network (CNN)-based framework is proposed to address challenges for navigation of an ES robot based on surgeons’ skill learning, and shows its capability of adapting to different situations and achieves similar success rate and average operating time.
Motion generation of robotic surgical tasks: Learning from expert demonstrations
- Computer Science2010 Annual International Conference of the IEEE Engineering in Medicine and Biology
- 2010
This paper proposes using programming by demonstration to build generative models and generate smooth trajectories that capture the underlying structure of the motion data recorded from expert demonstrations to generate smoother trajectories for reproduction of three common medical tasks.
Automating multi-throw multilateral surgical suturing with a mechanical needle guide and sequential convex optimization
- Computer Science, Medicine2016 IEEE International Conference on Robotics and Automation (ICRA)
- 2016
Initial results suggest that the dVRK can perform suturing at 30% of human speed while completing 86% suture throws attempted, and the Suture Needle Angular Positioner (SNAP) results in a 3x error reduction in the needle pose estimate in comparison with the standard actuator.
References
SHOWING 1-10 OF 28 REFERENCES
The Endo[PA]R system for minimally invasive robotic surgery
- Medicine2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566)
- 2004
This research project tackles haptic feedback and prolonged operation time in minimally invasive robotic surgery by adding high fidelity force-feedback and automating recurrent manipulation tasks into the Endo[PA]R (endoscopic partially-autonomous robot) system, an open evaluation platform for robotic surgery.
Robotic surgical instruments for dexterity enhancement in thoracoscopic coronary artery bypass graft.
- Medicine, EngineeringJournal of laparoendoscopic & advanced surgical techniques. Part A
- 1997
The use of robotic enhancement technology leads to an efficient performance of sutured coronary artery bypass anastomoses in a plastic model that enhances dexterity, precision, and reduces surgeon's fatigue while preserving the quality of hand suturing.
ROBOTIC ASSISTED SUTURING IN MINIMALLY INVASIVE SURGERY
- Medicine
- 2002
The focus of the research is the development of the EndoBot controller that is capable of a range of MIS operations including manual, shared, and supervised autonomous operations and particular emphasis is placed on its automation.
Manual vs robotically assisted laparoscopic surgery in the performance of basic manipulation and suturing tasks.
- MedicineArchives of surgery
- 1998
Laroscopic maneuvering and suturing is faster and just as precise when performed manually as when performed with the prototype robotic system, inversely proportional to the size of the suture.
Evolino: Hybrid Neuroevolution/Optimal Linear Search for Sequence Learning
- Computer ScienceIJCAI
- 2005
A general framework for sequence learning, EVOlution of recurrent systems with LINear outputs (Evolino), which uses evolution to discover good RNN hidden node weights, while using methods such as linear regression or quadratic programming to compute optimal linear mappings from hidden state to output.
Solving Non-Markovian Control Tasks with Neuro-Evolution
- Computer ScienceIJCAI
- 1999
This article demonstrates a neuroevolution system, Enforced Sub-populations (ESP), that is used to evolve a controller for the standard double pole task and a much harder, non-Markovian version, and introduces an incremental method that evolves on a sequence of tasks, and utilizes a local search technique (Delta-Coding) to sustain diversity.
Endo[PA]R: An Open Evaluation System for Minimally Invasive Robotic Surgery
- Medicine
- 2004
EndoPAR (Endoscopic Partial-Autonomous Robot) is an open experimental platform for robot assisted minimally invasive heart surgery with haptic feedback that was evaluated by surgeons and it was capable of performing automated knot-tying.
Efficient Reinforcement Learning through Symbiotic Evolution
- Computer ScienceMachine Learning
- 1996
A new reinforcement learning method called SANE (Symbiotic, Adaptive Neuro-Evolution), which evolves a population of neurons through genetic algorithms to form a neural network capable of performing a task, is presented.
Active Guidance for a Finless Rocket Using Neuroevolution
- Computer ScienceGECCO
- 2003
Results show that the evolved active guidance controller can greatly increase the final altitude of the rocket, and that ESP can be an effective method for solving real-world, non-linear control tasks.
Learning to Forget: Continual Prediction with LSTM
- Computer ScienceNeural Computation
- 2000
This work identifies a weakness of LSTM networks processing continual input streams that are not a priori segmented into subsequences with explicitly marked ends at which the network's internal state could be reset, and proposes a novel, adaptive forget gate that enables an LSTm cell to learn to reset itself at appropriate times, thus releasing internal resources.