Content-based processing and analysis of endoscopic images and videos: A survey

@article{Mnzer2016ContentbasedPA,
  title={Content-based processing and analysis of endoscopic images and videos: A survey},
  author={Bernd M{\"u}nzer and Klaus Sch{\"o}ffmann and L{\'a}szl{\'o} B{\"o}sz{\"o}rm{\'e}nyi},
  journal={Multimedia Tools and Applications},
  year={2016},
  volume={77},
  pages={1323-1362}
}
In recent years, digital endoscopy has established as key technology for medical screenings and minimally invasive surgery. Since then, various research communities with manifold backgrounds have picked up on the idea of processing and automatically analyzing the inherently available video signal that is produced by the endoscopic camera. Proposed works mainly include image processing techniques, pattern recognition, machine learning methods and Computer Vision algorithms. While most… 
View on Springer
link.springer.com
88 Citations
Highly Influential Citations
4
Background Citations
34
Methods Citations
14

Figures from this paper

88 Citations

Citation Type

Citation Type

A Survey on Object Detection, Annotation and Anomaly Detection Methods for Endoscopic Videos
TLDR
The objective is to understand the current research, methods and technology used for endoscopic video annotation and analysis which has been extrapolated to a detailed literature survey of automatic anomaly detection as well as object detection in the context of annotation of live endoscopic videos.
Semi-Automatic Retrieval of Relevant Segments from Laparoscopic Surgery Videos
TLDR
The thesis deals with the questions of how to semantically classify video frames using Convolutional Neural Networks into different semantic concepts of surgical actions and anatomical structures and investigates to what extent surgeons' needs are covered with the proposed extraction of relevant scenes.
Content-based retrieval of video segments from minimally invasive surgery videos using deep convolutional video descriptors and iterative query refinement
TLDR
This paper uses state-of-theart deep 3D convolutional neural network models pre-trained on large public video classification datasets to extract spatiotemporal features from MIS video segments and employs an iterative query refinement (IQR) strategy where in a support vector machine (SVM) classifier trained online based on relevance feedback from the user is used to refine the search results iteratively.
Endoscopy image restoration: A study of the kernel estimation from specular highlights
Efficient tool segmentation for endoscopic videos in the wild
TLDR
This work explores how to solve two of the most common challenges: real-time and memory restrictions and false positives in frames with no tools, by adapting an efficient general purpose semantic segmentation model.
Static vs. Dynamic Content Descriptors for Video Retrieval in Laparoscopy
TLDR
The experimental results show that although static descriptors achieve the highest overall performance, dynamic descriptors are much more discriminative for certain classes of surgical actions.
Video retrieval in laparoscopic video recordings with dynamic content descriptors
TLDR
This work proposes a video content descriptor called MIDD (Motion Intensity and Direction Descriptor) that can be effectively used to find similar segments in a laparoscopic video database and thereby help surgeons to more quickly inspect other instances of a given error scene.
A systematic review on content-based video retrieval
Algorithms for Real-Time Endoscopy Image Processing Pipeline in Clinical Decision Support Systems
TLDR
Aset ofｿalgorithms, £1.5m cost proposed for real-time realization of realized realization of “mosaic” Algorithm.
Multi-Granular Semantic Analysis Based on Nasal Endoscopic Video
TLDR
A multi-granularity semantic analysis framework that can simultaneously meet the accuracy and timeliness required for endoscopic surgery video semantic analysis is proposed and achieves a good performance in terms of accuracy and efficiency.
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 305 REFERENCES
Keyframe extraction in endoscopic video
TLDR
A new keyframe extraction approach for this special video domain is proposed and it is shown that this method is significantly better than a previously proposed approach.
Automatic real-time detection of endoscopic procedures using temporal features
Automatic real-time capture and segmentation of endoscopy video
TLDR
Novel software is presented that automates the capture of all images of a single colonoscopy into a single digitized video file that uses metrics based on color and motion over time to determine whether the images are derived from inside a single patient.
Endoscopic Video Retrieval: A Signature-Based Approach for Linking Endoscopic Images with Video Segments
TLDR
The proposed distance-based similarity model facilitates an adaptive representation of the visual properties of endoscopic images and allows for matching these properties efficiently and improves the current state of the art by one order of magnitude.
Classification approach for automatic laparoscopic video database organization
TLDR
The results show that SC with $$K$$K-singular value decomposition (K-SVD) yields the best classification accuracy and the end part of the laparoscopic videos is the most discriminative compared to the other parts of the videos.
Automatic Segmentation and Inpainting of Specular Highlights for Endoscopic Imaging
TLDR
A segmentation method based on nonlinear filtering and colour image thresholding and an efficient inpainting method that eliminates the negative effect of specular highlights on other image analysis algorithms and also gives a visually pleasing result are proposed.
Automatic feature generation in endoscopic images
TLDR
An automatic method that extracts features from images for use in classification that automatically computes good features, making it possible to classify endoscopic images into their respective surgical phases is developed.
Texture and color based image segmentation and pathology detection in capsule endoscopy videos
Computer-Aided Detection of Diagnostic and Therapeutic Operations in Colonoscopy Videos
TLDR
Based on new algorithms for instrument detection and shot segmentation, new spatio-temporal analysis techniques are introduced to automatically identify an operation shot - a segment of visual data in a colonoscopy video that corresponds to a diagnostic or therapeutic operation.
Segmentation of recorded endoscopic videos by detecting significant motion changes
TLDR
A new method for segmentation of endoscopic videos, based on spatial and temporal differences of motion in these videos, is proposed and it is shown that this approach provides reasonable performance.
...
1
2
3
4
5
...