Towards Perceptually Driven Segmentation Evaluation Metrics

  title={Towards Perceptually Driven Segmentation Evaluation Metrics},
  author={Elisa Drelie Gelasca and Touradj Ebrahimi and Myl{\`e}ne C. Q. Farias and Marco Carli and Sanjit K. Mitra},
  journal={2004 Conference on Computer Vision and Pattern Recognition Workshop},
To be reliable, an automatic segmentation evaluation metric has to be validated by subjective tests. In this paper, a formal protocol for subjective tests for segmentation quality assessment is presented. The most common artifacts produced by segmentation algorithms are identified and an extensive analysis of their effects on the perceived quality is performed. A psychophysical experiment was performed to assess the quality of video with segmentation errors. The results show how an objective… 

Figures and Tables from this paper

Performance Evaluation of Image Segmentation

A new approach for evaluation of segmentation is proposed that takes into account not only the accuracy of the boundary localization of the created segments but also the under-se segmentation and over-segmentation effects, regardless to the number of regions in each partition.

A human visual system-driven image segmentation algorithm

Adaptive Segmentation based on a Learned Quality Metric

Experimental results show an improved segmentation quality for the adaptive algorithm based on the segmentationquality model, which can be easily applied to a wide class of segmentation algorithms.

Image segmentation evaluation: A survey of unsupervised methods

Superiority Of Graph-Based Visual Saliency (GVS) Over Other Image Segmentation Methods

This study proposed a supervised framework which uses visual saliency detection to quantitatively evaluate image segmentation quality and uses Graph-based Visual Saliency (GVS) to compare boundary outputs for manually segmented images.

Designing evaluation methodologies: The case of motion detection

A motivated and comprehensive comparative evaluation methodology is described and used to compare two motion detection algorithms reported in the literature.

A survey of perceptual image processing methods

Performance evaluation in visual surveillance using the F-measure

The problems associated with optimising the operating point of motion detection algorithms and objective performance evaluation are explored, advantages of object-based approach are emphasised, issues associated with ROC optimisation are examined, and a solution based on F-measure optimisation is proposed.

Meta-Evaluation of Image Segmentation Using Machine Learning

A meta-evaluation method is proposed in which any set of base evaluation methods are combined by a machine learning algorithm that coalesces their evaluations based on a learned weighting function, which depends upon the image to be segmented.

An object-based comparative methodology for motion detection based on the F-Measure



Objective evaluation of segmentation masks in video sequences

In order to avoid the charge of visually evaluating the quality of segmentation masks, a procedure for evaluating this quality by means of algorithmically computed figures of merit is introduced.

Objective evaluation of video segmentation quality

Both stand alone and relative evaluation metrics are developed to cover the cases for which a reference segmentation is missing or available for comparison, as well as for complete segmentation partitions.

A survey on evaluation methods for image segmentation

A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics

A database containing 'ground truth' segmentations produced by humans for images of a wide variety of natural scenes is presented and an error measure is defined which quantifies the consistency between segmentations of differing granularities.

Metrics for performance evaluation of video object segmentation and tracking without ground-truth

Detailed results are presented to evaluate the segmentation map of the "Man" object in the "Hall Monitor" sequence both in terms of a single numerical measure, as well as localization of the good and bad segments of the boundary.

Performance evaluation metrics for object-based video segmentation

The proposed metrics are used to evaluate three essentially different approaches for video segmentation, i.e., an edge-based, a motion clusteringbased, and a total feature vector clustering based algorithm.

Robust Visual Method for Assessing the Relative Performance of Edge-Detection Algorithms

A new method for evaluating edge detection algorithms is presented and applied to measure the relative performance of algorithms by Canny, Nalwa-Binford, Iverson-Zucker, Bergholm, and Rothwell, and the results agree with visual evaluations of the edge images.

Intuitive strategy for parameter setting in video segmentation

Simulation results on various test sequences show that optimal set of parameters can be obtained efficiently and largely improve the results obtained when compared to a simple implementation of the same segmentation algorithm with ad-hoc parameter setting strategy.

Comparing Images Using the Hausdorff Distance

Efficient algorithms for computing the Hausdorff distance between all possible relative positions of a binary image and a model are presented and it is shown that the method extends naturally to the problem of comparing a portion of a model against an image.

Objective evaluation of segmentation quality using spatio-temporal context

An automatic method based on computing the deviation of the segmentation results from a reference segmentation based on spatial and temporal contextual information, by taking into account the way humans perceive visual information is proposed.