Augmented reality with Microsoft HoloLens holograms for near infrared fluorescence based image guided surgery

@inproceedings{Cui2017AugmentedRW,
  title={Augmented reality with Microsoft HoloLens holograms for near infrared fluorescence based image guided surgery},
  author={Nan Cui and Pradosh Kharel and V. Gruev},
  booktitle={BiOS},
  year={2017}
}
Near infrared fluorescence (NIRF) based image guided surgery aims to provide vital information to the surgeon in the operating room, such as locations of cancerous tissue that should be resected and healthy tissue that should to be preserved. Targeted molecular markers, such as tumor or nerve specific probes, are used in conjunctions with NIRF imaging and display systems to provide key information to the operator in real-time. One of the major hurdles for the wide adaptation of these imaging… Expand
Towards X-ray free endovascular interventions – using HoloLens for on-line holographic visualisation
TLDR
The authors present the development of a real-time navigation framework, which allows a 3D holographic view of the vascular system without any need of radiation, and shows great potential to improve the workflow for endovascular procedures, by simultaneously reducing X-ray exposure. Expand
A HoloLens Framework for Augmented Reality Applications in Breast Cancer Surgery
This project aims to support oncologic breast-conserving surgery by creating a platform for better surgical planning through the development of a framework that is capable of displaying a virtualExpand
Registration Techniques for Clinical Applications of Three-Dimensional Augmented Reality Devices
Many clinical procedures would benefit from direct and intuitive real-time visualization of anatomy, surgical plans, or other information crucial to the procedure. Three-dimensional augmented realityExpand
Through the HoloLens™ looking glass: augmented reality for extremity reconstruction surgery using 3D vascular models with perforating vessels
TLDR
This work has demonstrated that AR can assist the accurate identification, dissection and execution of vascular pedunculated flaps during reconstructive surgery. Expand
Head-mounted display augmented reality to guide pedicle screw placement utilizing computed tomography
TLDR
This is the first peer-reviewed report and evaluation of HMD-AR with superimposed 3D guidance utilizing CT for spinal pedicle guide placement for the purpose of cannulation without the use of fluoroscopy. Expand
Visual Tracking vs Optical Tracking in Computer-Assisted Intervention
Computed-Assisted Intervention (CAI) aims to safely guide the surgeon during surgical interventions, which typically relies on Optical Tracking (OT) systems to provide the location of tools andExpand
Kidney edge detection in laparoscopic image data for computer-assisted surgery
TLDR
This work proposes a deep learning-based solution that succeeds in predicting the edges of the kidney, except in instances where high occlusion occurs, which explains the average decrease in the IOU score. Expand
Angular Dependence of the Spatial Resolution in Virtual Reality Displays
We compare two methods for characterizing the angular dependence of the spatial resolution in virtual reality head-mounted displays (HMDs) by measuring the line spread response (LSR) across the fieldExpand
Augmented Reality as a Telemedicine Platform for Remote Procedural Training
TLDR
The HoloLens is used to capture the first-person view of a simulated rural emergency room (ER) through mixed reality capture (MRC) and serves as a novel telemedicine platform with remote pointing capabilities and compares the findings to that of a more traditional multi-camera telemedICine solution. Expand
Design of Si3N4 waveguides and components to form an integrated optical network for retinal projection in thin augmented reality glasses
TLDR
This work presents a detailed description of the optical principles behind the retinal projection for augmented reality (AR) glasses, including the self-focusing effect, and presents the design of the first building blocks used for the optical integrated circuit at a visible wavelength. Expand
...
1
2
3
...

References

SHOWING 1-10 OF 13 REFERENCES
Image-guided cancer surgery using near-infrared fluorescence
TLDR
The concept of NIR fluorescence imaging for cancer surgery is introduced, the clinical trial literature to date is examined, and the key issues pertaining to imaging system and contrast agent optimization are outlined. Expand
Binocular Goggle Augmented Imaging and Navigation System provides real-time fluorescence image guidance for tumor resection and sentinel lymph node mapping
TLDR
Clinical use of the GAINS to guide tumor resection and sentinel lymph node mapping promises to improve surgical outcomes, reduce rates of repeat surgery, and improve the accuracy of cancer staging. Expand
The FLARE™ Intraoperative Near-Infrared Fluorescence Imaging System: A First-in-Human Clinical Trial in Breast Cancer Sentinel Lymph Node Mapping
TLDR
Simultaneous pan-lymphatic and SLN mapping was demonstrated in swine using clinically available NIR fluorophores and the dual NIR capabilities of the FLARE™ system, describing the successful clinical translation of a new NIR fluorescence imaging system for image-guided oncologic surgery. Expand
The clinical use of indocyanine green as a near‐infrared fluorescent contrast agent for image‐guided oncologic surgery
TLDR
Although non‐targeted and non‐conjugatable, ICG appears to be laying the foundation for more widespread use of NIR fluorescence‐guided surgery, understanding its advantages and limitations is of significant importance. Expand
Intraoperative near infrared fluorescence guided identification of the ureters using low dose methylene blue: a first in human experience.
TLDR
This study demonstrates the first successful use of near infrared fluorescence using low dose methylene blue for the identification of the ureters during lower abdominal surgery. Expand
Molecular imaging with optics: primer and case for near-infrared fluorescence techniques in personalized medicine.
TLDR
This work compares and contrast the development of optical molecular imaging techniques with nuclear medicine with a didactic emphasis for initiating readers into the field of molecular imaging and comments on the challenges and progress for translating near-infrared molecular imaging agents for personalized medicine. Expand
Comparison of visible and near-infrared wavelength-excitable fluorescent dyes for molecular imaging of cancer.
TLDR
Comparison of the in vivo fluorescent imaging performance of a near-IR (NIR) and a red-excitable dye is reported, showing that while both agents target EGFr, the EGF-IRDye 800CW evidenced a significantly reduced background and enhanced the tumor-to-background ratio (TBR) compared to the E GF-Cy5.5. Expand
Intraoperative identification of sentinel lymph nodes by near-infrared fluorescence imaging in patients with breast cancer.
TLDR
A novel method for sentinel lymph node (SLN) identification by fluorescence imaging that provides a high detection rate and a low false-negativity rate and is feasible and safe for intraoperative detection of SLN allowing real-time observation without any need for training is presented. Expand
SPY: an innovative intra-operative imaging system to evaluate graft patency during off-pump coronary artery bypass grafting.
TLDR
SPY images provide critical information to surgeons to detect non-patent grafts, allowing them to be revised while the patient is still on the operating table, and may become the gold standard for surgical management in the near future. Expand
Fluorescence properties and metabolic features of indocyanine green (ICG) as related to angiography.
TLDR
The focus of this paper is to review the basic properties of ICG and to show how some clinical features related to basic properties also depend on the instrumentation used to perform ICG angiography. Expand
...
1
2
...