Understanding the reliability of citizen science observational data using item response models

  title={Understanding the reliability of citizen science observational data using item response models},
  author={Edgar Santos‐Fernandez and Kerrie Lee Mengersen},
  journal={Methods in Ecology and Evolution},
  pages={1533 - 1548}
Citizen science projects have become increasingly popular in many fields, including ecology. However, the quality of this information is frequently debated within the scientific community. Modern citizen science implementations therefore require measures of the users' proficiency. We introduce a new methodological framework of item response that quantifies a citizen scientist's ability, taking into account the difficulty of the task. We focus on citizen science programs involving the… 



Assessing data quality in citizen science

Each citizen-science dataset should be judged individually, according to project design and application, and not assumed to be substandard simply because volunteers generated it.

Statistical solutions for error and bias in global citizen science datasets

Occupancy models for citizen‐science data

Large‐scale citizen‐science projects, such as atlases of species distribution, are an important source of data for macroecological research, for understanding the effects of climate change and other

Estimates of observer expertise improve species distributions from citizen science data

The results highlight the value of recording observer identity and other detectability covariates, to control for sources of bias associated with the observation process and may be most important where citizen science schemes encourage wide participation.

Correcting misclassification errors in crowdsourced ecological data: A Bayesian perspective

It is demonstrated how Bayesian hierarchical models can be used to learn about latent variables of interest, while accounting for the participants’ abilities, in an ecological application that involves crowdsourced classifications of georeferenced coral reef images from the Great Barrier Reef, Australia.

Can Observation Skills of Citizen Scientists Be Estimated Using Species Accumulation Curves?

A method for indexing observer variability based on the data routinely submitted by observers participating in the citizen science project eBird, a broad-scale monitoring project in which observers collect and submit lists of the bird species observed while birding, finds that differences in species accumulation curves among observers equates to higher rates of species accumulation.

Modelling imperfect presence data obtained by citizen science

There is growing awareness about the potential benefit of harnessing citizen science for research, particularly in the biological and environmental sciences. Data quality is a major constraint in the

Improving big citizen science data: Moving beyond haphazard sampling

This paper argues that the haphazard structure of the data has been seen as an unfortunate but unchangeable aspect of citizen science data, and provides a very simple, tractable framework that could be adapted by broadscale citizen science projects to allow citizen scientists to optimize the marginal value of their efforts.

Optimizing future biodiversity sampling by citizen scientists

A novel framework which assigns every citizen science sampling event a marginal value, derived from the importance of an observation to the understanding of overall population trends is developed, and it is found that past observations are useful in forecasting where high-value observations will occur in the future.

Efficient occupancy model-fitting for extensive citizen-science data

Motivated by presence-only records of occurrence from the UK Butterflies for the New Millennium data base, this work presents an alternative approach, in which site variation is described in a standard way through logistic regression on relevant environmental covariates, which allows efficient occupancy model-fitting using classical inference.