Build, Compute, Critique, Repeat: Data Analysis with Latent Variable Models

@inproceedings{Blei2014BuildCC,
  title={Build, Compute, Critique, Repeat: Data Analysis with Latent Variable Models},
  author={David M. Blei},
  year={2014}
}
  • D. Blei
  • Published 3 January 2014
  • Computer Science
We survey latent variable models for solving data-analysis problems. A latent variable model is a probabilistic model that encodes hidden patterns in the data. We uncover these patterns from their conditional distribution and use them to summarize data and form predictions. Latent variable models are important in many fields, including computational biology, natural language processing, and social network analysis. Our perspective is that models are developed iteratively: We build a model, use… 

Figures from this paper

Scalable inference of discrete data: User behavior, networks and genetic variation
TLDR
A Bayesian nonparametric model is developed in which the latent representations of users and items grow to accommodate new data, and novel algorithms for discovering overlapping communities in large networks are developed.
Do-calculus enables causal reasoning with latent variable models
TLDR
It is demonstrated that an LVM can answer any causal query posed post-training, provided that the query can be identified from the observed variables according to the do-calculus rules.
Latent Factor Regressions for the Social Sciences
TLDR
It is shown that interactive latent factor models provide a powerful modeling alternative that can address a wide range of data types and introduce a class of fast variational inference algorithms that allows for models to be fit quickly and accurately.
Predictive learning as a network mechanism for extracting low-dimensional latent space representations
TLDR
This work investigates the hypothesis that a means for generating representations with easily accessed low-dimensional latent structure is through learning to predict observations about the world, and investigates whether and when network mechanisms for sensory prediction coincide with those for extracting the underlying latent variables.
Efficient Marginalization of Discrete and Structured Latent Variables via Sparsity
TLDR
A new training strategy is proposed which parameterize discrete distributions over latent assignments using differentiable sparse mappings: sparsemax and its structured counterparts, which enables efficient marginalization.
Dynamic Poisson Factorization
TLDR
dPF, a dynamic matrix factorization model based on the recent Poisson factorizationmodel for recommendations, is proposed, which models the time evolving latent factors with a Kalman filter and the actions with Poisson distributions.
Diversity-Promoting Bayesian Learning of Latent Variable Models
TLDR
This paper proposes a diversity-promoting mutual angular prior which assigns larger density to components with larger mutual angles and uses this prior to affect the posterior via Bayes' rule, and develops two efficient approximate posterior inference algorithms based on variational inference and MCMC sampling.
Latent Variable Modeling with Diversity-Inducing Mutual Angular Regularization
TLDR
A novel regularization technique for LVMs is developed, which controls the geometry of the latent space during learning to enable the learned latent components of LVMs to be diverse in the sense that they are favored to be mutually different from each other, to accomplish long-tail coverage, low redundancy, and better interpretability.
Search for K: Assessing Five Topic-Modeling Approaches to 120,000 Canadian Articles
TLDR
Mixed findings from this research complement advances in topic modeling and provide insights into the choice of optimal topics in social science research.
Variational Inference over Nonstationary Data Streams for Exponential Family Models
TLDR
This paper makes use of a novel scheme based on hierarchical priors to explicitly model temporal changes of the model parameters, and shows how this approach induces an exponential forgetting mechanism with adaptive forgetting rates.
...
...

References

SHOWING 1-10 OF 130 REFERENCES
Inferring Parameters and Structure of Latent Variable Models by Variational Bayes
TLDR
The Variational Bayes framework is presented, which approximates full posterior distributions over model parameters and structures, as well as latent variables, in an analytical manner without resorting to sampling methods, and can be applied to a large class of models in several domains.
Approximate Bayesian inference for latent Gaussian models by using integrated nested Laplace approximations
TLDR
This work considers approximate Bayesian inference in a popular subset of structured additive regression models, latent Gaussian models, where the latent field is Gaussian, controlled by a few hyperparameters and with non‐Gaussian response variables and can directly compute very accurate approximations to the posterior marginals.
Graphical Models, Exponential Families, and Variational Inference
TLDR
The variational approach provides a complementary alternative to Markov chain Monte Carlo as a general source of approximation methods for inference in large-scale statistical models.
Stochastic variational inference
TLDR
Stochastic variational inference lets us apply complex Bayesian models to massive data sets, and it is shown that the Bayesian nonparametric topic model outperforms its parametric counterpart.
Multiple Imputation for Model Checking: Completed‐Data Plots with Missing and Latent Data
TLDR
The methods of missing‐data model checking can be interpreted as “predictive inference” in a non‐Bayesian context and the graphical diagnostics within this framework are considered.
Mixed Membership Stochastic Blockmodels
TLDR
This paper describes a latent variable model of such data called the mixed membership stochastic blockmodel, which extends blockmodels for relational data to ones which capture mixed membership latent relational structure, thus providing an object-specific low-dimensional representation.
An Introduction to Variational Methods for Graphical Models
TLDR
This paper presents a tutorial introduction to the use of variational methods for inference and learning in graphical models (Bayesian networks and Markov random fields), and describes a general framework for generating variational transformations based on convex duality.
Probabilistic topic models
  • D. Blei
  • Computer Science
    Commun. ACM
  • 2010
TLDR
Surveying a suite of algorithms that offer a solution to managing large document archives suggests they are well-suited to handle large amounts of data.
Probabilistic Graphical Models - Principles and Techniques
TLDR
The framework of probabilistic graphical models, presented in this book, provides a general approach for causal reasoning and decision making under uncertainty, allowing interpretable models to be constructed and then manipulated by reasoning algorithms.
Variational inference in nonconjugate models
TLDR
These methods allow for easily derived variational algorithms with a wide class of nonconjugate models; they extend and unify some of the existing algorithms that have been derived for specific models; and they work well on real-world data sets.
...
...