Performance Metrics: How and When

@article{Israel2006PerformanceMH,
  title={Performance Metrics: How and When},
  author={Steven A. Israel},
  journal={Geocarto International},
  year={2006},
  volume={21},
  pages={23 - 32}
}
  • S. Israel
  • Published 1 June 2006
  • Mathematics
  • Geocarto International
Abstract This paper compares different classification performance metrics commonly used for environmental monitoring with remotely sensed satellite data and their associated error bounds. Discriminant functions were generated for three classifiers using common training data. Identical independent test data were classified by each discriminant function. The classified test values were evaluated by each performance metric. Each performance measure, its error bound, and its significance were… 
View on Taylor & Francis
8 Citations
Background Citations
4

8 Citations

Citation Type

Citation Type

Comparing the Effects of Annotation Type on Machine Learning Detection Performance
TLDR
This paper evaluates the performance tradeoffs of three alternative methods of annotating imagery for use in ML and compares the detection accuracy of and differences between the resultant models.
Vision physiology applied to hyperspectral short wave infrared imaging
TLDR
This paper explores using spectral response functions for the Short Wave Infrared (SWIR) region that are not sharp, but broad and overlapping and even more complex than those found in the retina for the visible region.
Context Assumptions for Threat Assessment Systems
TLDR
The intent of this chapter is to enable system users to characterize trust using the decision support system by establishing the context of the decision by categorizing six assumptions: specific problem, acquirable data, use of context, reproducible analysis, actionable intelligence, and quantifiable decision making.
Toward a common lexicon for exploiting activity data
  • S. Israel
  • Computer Science
    2012 IEEE Applied Imagery Pattern Recognition Workshop (AIPR)
  • 2012
TLDR
A lexicon that has been developed from the pattern recognition community and applies it to the current tasks of social network interactions is provided.
Heartbeat biometrics: a sensing system perspective
TLDR
The research indicates that the heartbeat contains rich information about the individual, their level of anxiety, and the cardiac state.
Generative Adversarial Networks for Classification
TLDR
The use of GANs is expanded from detection (k=2) to classification (k>2) problems, with results showing no significant performance differences among any of the network configurations or training set size trials.
Assessing and tracing the outcomes and impact of research infrastructures
TLDR
This article discusses how research infrastructures are identified and referenced by scholars in the research literature and how those references are being collected and analyzed for the purposes of evaluating impact and identifies notable challenges that impede the analysis of impact metrics.
Evaluation of ISR technologies for counter insurgency warfare
  • S. Israel
  • Sociology
    2010 IEEE 39th Applied Imagery Pattern Recognition Workshop (AIPR)
  • 2010
TLDR
This paper compares and contrasts strategies used to test technologies for the Cold War era warfare to the test strategies employed for counter-insurgency warfare, showing the change from technologies that describe what-where to technologies that additionally describe activities.

References

SHOWING 1-10 OF 33 REFERENCES
Tau coefficients for accuracy assessment of classification of remote sensing data
The Kappa coefficient is generally used to assess the accuracy of image classifications. We introduce the Tau coefficient, which measures the improvement of a classification over a random assignment…
A coefficient of agreement as a measure of thematic classification accuracy.
The classification error matrix typically contains tabulated results of accuracy evaluation for a thematic classification, such as a land-use and land-cover map. Diagonal elements of the matrix…
A multinomial selection procedure for evaluating pattern recognition algorithms
Status of land cover classification accuracy assessment
Discriminant Analysis and Statistical Pattern Recognition
Provides a systematic account of the subject area, concentrating on the most recent advances in the field. While the focus is on practical considerations, both theoretical and practical issues are…
On the classification of multispectral satellite images using the multilayer perceptron
Assessing the accuracy of remotely sensed data : principles and practices
TLDR
This chapter discusses Accuracy Assessment, which examines the impact of sample design on cost, statistical Validity, and measuring Variability in the context of data collection and analysis.
A land-use classification system for use with remote sensor data
New demands on our land resources require more stringent controls and management practices. The administration of these controls requires better and more frequent information concerning land use.…
The effect of training set size and composition on artificial neural network classification
TLDR
The results showed that higher classification accuracies were generally derived from the artificial neural network, especially when small training sets only were available, and it was apparent that the opportunity of the artificial Neural Network to learn class appearance was influenced by the composition of the training set.
On the compensation for chance agreement in image classification accuracy assessment, Photogram
TLDR
Two kappa-like approaches which compensate more appropriately for the degree of chance agreement are discussed, and they may be more suitable for the assessment of image classification accuracy than the kappa coefficient.
...
1
2
3
4
...