conftrace_

2012 NeurIPS NeurIPS 2012

Learning from the Wisdom of Crowds by Minimax Entropy

Abstract

An important way to make large training sets is to gather noisy labels from crowds of nonexperts. We propose a minimax entropy principle to improve the quality of these labels. Our method assumes that labels are generated by a probability distribution over workers, items, and labels. By maximizing the entropy of this distribution, the method naturally infers item confusability and worker expertise. We infer the ground truth by minimizing the entropy of this distribution, which we show minimizes the Kullback-Leibler (KL) divergence between the probability distribution and the unknown truth. We show that a simple coordinate descent scheme can optimize minimax entropy. Empirically, our results are substantially better than previously published methods for the same problem.

🐣 Hot Topic Early Bird - noisy label

Authors

Dengyong Zhou, Sumit Basu, Yi Mao, John C. Platt

Topics

Artificial Intelligence > Bayesian & Probabilistic > Probabilistic Modeling Machine Learning > Core Methods > Classification Machine Learning > Learning Types > Unsupervised Learning Machine Learning > Learning Types > Weakly Supervised Learning Machine Learning > Optimization & Theory > Loss Functions Machine Learning > Bayesian & Probabilistic > Probabilistic Modeling Machine Learning > Learning Types > Label Noise Machine Learning > Learning Types > Crowdsourcing

Keywords

probabilistic modeling label noise kullback-leibler divergence coordinate descent label aggregation crowd sourcing crowdsourced learning minimax entropy worker expertise noisy label crowdsourced label crowd labeling ground truth inference

More from NeurIPS 2012

All NeurIPS 2012 papers →

Sparse Approximate Manifolds for Differential Geometric MCMC

Bayesian active learning with localized priors for fast receptive field characterization

Deep Neural Networks Segment Neuronal Membranes in Electron Microscopy Images

Parametric Local Metric Learning for Nearest Neighbor Classification

The topographic unsupervised learning of natural sounds in the auditory cortex

Efficient Sampling for Bipartite Matching Problems