Resampling methods for evaluating classification accuracy of wildlife habitat models

@article{Verbyla1989ResamplingMF,
  title={Resampling methods for evaluating classification accuracy of wildlife habitat models},
  author={David L. Verbyla and John A. Litvaitis},
  journal={Environmental Management},
  year={1989},
  volume={13},
  pages={783-787}
}
Predictive models of wildlife-habitat relationships often have been developed without being tested The apparent classification accuracy of such models can be optimistically biased and misleading. Data resampling methods exist that yield a more realistic estimate of model classification accuracy These methods are simple and require no new sample data. We illustrate these methods (cross-validation, jackknife resampling, and bootstrap resampling) with computer simulation to demonstrate the… 
Predictive Models of Fish Species Distributions: A Note on Proper Validation and Chance Predictions
TLDR
It is demonstrated that the commonly employed resubstitution approach to model validation produces highly biased estimates of correct classification rates and consequently an inaccurate perception of true model performance.
Computer-intensive methods in the analysis of species-habitat relationships
TLDR
Two procedures for constraining model complexity { backward stepwise variable selection and penalized maximum likelihood} are investigated and internal validation via bootstrap resampling improved the estimates of model performance.
A review of methods for the assessment of prediction errors in conservation presence/absence models
TLDR
Thirteen recommendations are made to enable the objective selection of an error assessment technique for ecological presence/absence models and a new approach to estimating prediction error, which is based on the spatial characteristics of the errors, is proposed.
Internal Validation of Predictive Logistic Regression Models for Decision-Making in Wildlife Management
TLDR
Using the data on successfully breeding wolf packs in the northern Rocky Mountains, it is validated that predictions from a model developed with data specific to each of six analysis areas are better calibrated to each population than a global model developed using all data simultaneously.
Assessing Habitat Suitability Models for the Deep Sea: Is Our Ability to Predict the Distributions of Seafloor Fauna Improving?
Methods that predict the distributions of species and habitats by developing statistical relationships between observed occurrences and environmental gradients have become common tools in
Predicting habitat suitability for wildlife in southeastern Arizona using Geographic Information Systems: scaled quail, a case study
Studies have used Geographic Information Systems (GIS) to evaluate habitat suitability for wildlife on a landscape scale, yet few have established the accuracy of these models. Based on documented
Informed statistical modelling of habitat suitability for rare and threatened species
In this thesis a number of statistical methods have been developed and applied to habitat suitability modelling for rare and threatened species. Data available on these species are typically limited.
Effects of sample size on accuracy of species distribution models
Effects of species’ ecology on the accuracy of distribution models
TLDR
None of the ecological traits tested provides an obvious correlate for environmental niche breadth or intra-specific niche differentiation, and these analyses provide conservation scientists and resource managers with a rule of thumb that helps distinguish between species whose occurrence is reliably or less reliably predicted by distribution models.
...
...

References

SHOWING 1-10 OF 13 REFERENCES
Questionable multivariate statistical inference in wildlife habitat and community studies (a reply)
We analyzed a data set constructed from functionally unrelated, easily collected observations (e.g., meat, stock, and liquor prices) around Fort Collins, Colorado, using principal components analysis
How Biased is the Apparent Error Rate of a Prediction Rule
TLDR
This work provides simple estimates for the downward bias of the apparent error rate of logistic regression on binary data, with error rates measured by the proportion of misclassified cases.
Influence of Understory Characteristics on Snowshoe Hare Habitat Use and Density
TLDR
The influence of forest understory characteristics on snowshoe hare habitat use and density was studied in eastern (Cherryfield) and western (Pierce Pond) Maine during 1981-83.
Bootstrap Techniques for Error Estimation
TLDR
Results on the application of several bootstrap techniques in estimating the error rate of 1-NN and quadratic classifiers show that, in most cases, the confidence interval of a bootstrap estimator of classification error is smaller than that of the leave-one-out estimator.
Estimating the Error Rate of a Prediction Rule: Improvement on Cross-Validation
Abstract We construct a prediction rule on the basis of some data, and then wish to estimate the error rate of this rule in classifying future observations. Cross-validation provides a nearly
Use of discriminant function to characterize ruffed grouse drumming sites in Georgia: a critique.
TLDR
The univariate statistical comparisons and discriminant function analysis used to identify the most important characteristics of ruffed grouse drumming sites in Georgia were backed by sound biological discussion.
Estimation of Error Rates in Discriminant Analysis
Several methods of estimating error rates in Discriminant Analysis are evaluated by sampling methods. Multivariate normal samples are generated on a computer which have various true probabilities of
Cross-Validation of Regression Models
TLDR
Attention is given to models obtained via subset selection procedures, which are extremely difficult to evaluate by standard techniques, and their use illustrated in examples.
Classification and Regression Trees
TLDR
This chapter discusses tree classification in the context of medicine, where right Sized Trees and Honest Estimates are considered and Bayes Rules and Partitions are used as guides to optimal pruning.
...
...