Bayesian Heatmaps: Probabilistic Classification with Multiple Unreliable Information Sources

  title={Bayesian Heatmaps: Probabilistic Classification with Multiple Unreliable Information Sources},
  author={Edwin Simpson and Steven Reece and Stephen J. Roberts},
Unstructured data from diverse sources, such as social media and aerial imagery, can provide valuable up-to-date information for intelligent situation assessment. Mining these different information sources could bring major benefits to applications such as situation awareness in disaster zones and mapping the spread of diseases. Such applications depend on classifying the situation across a region of interest, which can be depicted as a spatial “heatmap”. Annotating unstructured data using… 
BCCNet: Bayesian classifier combination neural network
BCCNet is described, a framework that simultaneously aggregates biased and contradictory labels from the crowd and trains an automatic classifier to process new data and shows the efficacy of the method in the challenging context of developing world applications.
Scalable Bayesian preference learning for crowds
A scalable Bayesian preference learning method for jointly predicting the preferences of individuals as well as the consensus of a crowd from pairwise labels is proposed, using a Bayesian approach to account for uncertainty arising from noisy and sparse data.
A Review of Judgment Analysis Algorithms for Crowdsourced Opinions
A comprehensive overview of the judgment analysis problem and some of its novel variants, addressed with different approaches, where the opinions are crowdsourced is provided.


Bayesian nonparametric crowdsourcing
Two new fully unsupervised models based on a Chinese restaurant process (CRP) prior and a hierarchical structure that allows inferring these groups jointly with the ground truth and the properties of the users are proposed.
Crowdsourcing Spatial Phenomena Using Trust-Based Heteroskedastic Gaussian Processes
This work uses a heteroskedastic Gaussian process model to incorporate user trust modelling into Bayesian spatial regression and is able to estimate the spatial function at any location of interest and also learn the level of trustworthiness of each user.
Community-based bayesian aggregation models for crowdsourcing
A novel community-based Bayesian label aggregation model, CommunityBCC, which assumes that crowd workers conform to a few different types, where each type represents a group of workers with similar confusion matrices, and consistently outperforms state-of-the-art label aggregation methods.
A Joint Gaussian Process Model for Active Visual Recognition with Expertise Estimation in Crowdsourcing
We present a noise resilient probabilistic model for active learning of a Gaussian process classifier from crowds, i.e., a set of noisy labelers. It explicitly models both the overall label noise and
Language Understanding in the Wild: Combining Crowdsourcing and Machine Learning
This work presents a novel Bayesian approach to language understanding that relies on aggregated crowdsourced judgements, and encodes the relationships between labels and text features in documents, such as tweets, web articles, and blog posts, accounting for the varying reliability of human labellers.
Determining intent using hard/soft data and Gaussian process classifiers
A mathematical framework for dealing with diverse data by representing it in the common format of a kernel matrix and irrelevant detail by automatically detecting the relevant (or, conversely, the irrelevant) components within a multi-source dataset is presented.
Eliminating Spammers and Ranking Annotators for Crowdsourced Labeling Tasks
An empirical Bayesian algorithm called SpEM is proposed that iteratively eliminates the spammers and estimates the consensus labels based only on the good annotators and is motivated by defining a spammer score that can be used to rank the annotators.
Dynamic Bayesian Combination of Multiple Imperfect Classifiers
This chapter explores Bayesian classifier combination, using the computationally efficient framework of variational Bayesian inference, and applies the approach to real data from a large citizen science project, Galaxy Zoo Supernovae, and shows that it far outperforms other established approaches to imperfect decision combination.
Time-Sensitive Bayesian Information Aggregation for Crowdsourcing Systems
A new time-sensitive Bayesian aggregation method that simultaneously estimates a task's duration and obtains reliable aggregations of crowdsourced judgments, called BCCTime.
Bayesian Classifier Combination
A general framework for Bayesian model combination (which differs from model averaging) in the context of classification is explored, which explicitly models the relationship between each model’s output and the unknown true label.