# Build, Compute, Critique, Repeat: Data Analysis with Latent Variable Models

@inproceedings{Blei2014BuildCC, title={Build, Compute, Critique, Repeat: Data Analysis with Latent Variable Models}, author={David M. Blei}, year={2014} }

We survey latent variable models for solving data-analysis problems. A latent variable model is a probabilistic model that encodes hidden patterns in the data. We uncover these patterns from their conditional distribution and use them to summarize data and form predictions. Latent variable models are important in many fields, including computational biology, natural language processing, and social network analysis. Our perspective is that models are developed iteratively: We build a model, use…

## 269 Citations

Scalable inference of discrete data: User behavior, networks and genetic variation

- Computer Science
- 2015

A Bayesian nonparametric model is developed in which the latent representations of users and items grow to accommodate new data, and novel algorithms for discovering overlapping communities in large networks are developed.

Do-calculus enables causal reasoning with latent variable models

- Computer ScienceArXiv
- 2021

It is demonstrated that an LVM can answer any causal query posed post-training, provided that the query can be identified from the observed variables according to the do-calculus rules.

Latent Factor Regressions for the Social Sciences

- Computer Science
- 2014

It is shown that interactive latent factor models provide a powerful modeling alternative that can address a wide range of data types and introduce a class of fast variational inference algorithms that allows for models to be fit quickly and accurately.

Predictive learning as a network mechanism for extracting low-dimensional latent space representations

- Computer Science, PsychologyNature communications
- 2021

This work investigates the hypothesis that a means for generating representations with easily accessed low-dimensional latent structure is through learning to predict observations about the world, and investigates whether and when network mechanisms for sensory prediction coincide with those for extracting the underlying latent variables.

Efficient Marginalization of Discrete and Structured Latent Variables via Sparsity

- Computer ScienceNeurIPS
- 2020

A new training strategy is proposed which parameterize discrete distributions over latent assignments using differentiable sparse mappings: sparsemax and its structured counterparts, which enables efficient marginalization.

Dynamic Poisson Factorization

- Computer ScienceRecSys
- 2015

dPF, a dynamic matrix factorization model based on the recent Poisson factorizationmodel for recommendations, is proposed, which models the time evolving latent factors with a Kalman filter and the actions with Poisson distributions.

Diversity-Promoting Bayesian Learning of Latent Variable Models

- Computer ScienceICML
- 2016

This paper proposes a diversity-promoting mutual angular prior which assigns larger density to components with larger mutual angles and uses this prior to affect the posterior via Bayes' rule, and develops two efficient approximate posterior inference algorithms based on variational inference and MCMC sampling.

Latent Variable Modeling with Diversity-Inducing Mutual Angular Regularization

- Computer ScienceArXiv
- 2015

A novel regularization technique for LVMs is developed, which controls the geometry of the latent space during learning to enable the learned latent components of LVMs to be diverse in the sense that they are favored to be mutually different from each other, to accomplish long-tail coverage, low redundancy, and better interpretability.

Search for K: Assessing Five Topic-Modeling Approaches to 120,000 Canadian Articles

- Computer Science2019 IEEE International Conference on Big Data (Big Data)
- 2019

Mixed findings from this research complement advances in topic modeling and provide insights into the choice of optimal topics in social science research.

Variational Inference over Nonstationary Data Streams for Exponential Family Models

- Computer ScienceMathematics
- 2020

This paper makes use of a novel scheme based on hierarchical priors to explicitly model temporal changes of the model parameters, and shows how this approach induces an exponential forgetting mechanism with adaptive forgetting rates.

## References

SHOWING 1-10 OF 130 REFERENCES

Inferring Parameters and Structure of Latent Variable Models by Variational Bayes

- Computer ScienceUAI
- 1999

The Variational Bayes framework is presented, which approximates full posterior distributions over model parameters and structures, as well as latent variables, in an analytical manner without resorting to sampling methods, and can be applied to a large class of models in several domains.

Approximate Bayesian inference for latent Gaussian models by using integrated nested Laplace approximations

- Computer Science
- 2009

This work considers approximate Bayesian inference in a popular subset of structured additive regression models, latent Gaussian models, where the latent field is Gaussian, controlled by a few hyperparameters and with non‐Gaussian response variables and can directly compute very accurate approximations to the posterior marginals.

Graphical Models, Exponential Families, and Variational Inference

- Computer ScienceFound. Trends Mach. Learn.
- 2008

The variational approach provides a complementary alternative to Markov chain Monte Carlo as a general source of approximation methods for inference in large-scale statistical models.

Stochastic variational inference

- Computer ScienceJ. Mach. Learn. Res.
- 2013

Stochastic variational inference lets us apply complex Bayesian models to massive data sets, and it is shown that the Bayesian nonparametric topic model outperforms its parametric counterpart.

Multiple Imputation for Model Checking: Completed‐Data Plots with Missing and Latent Data

- Computer ScienceBiometrics
- 2005

The methods of missing‐data model checking can be interpreted as “predictive inference” in a non‐Bayesian context and the graphical diagnostics within this framework are considered.

Mixed Membership Stochastic Blockmodels

- Computer ScienceNIPS
- 2008

This paper describes a latent variable model of such data called the mixed membership stochastic blockmodel, which extends blockmodels for relational data to ones which capture mixed membership latent relational structure, thus providing an object-specific low-dimensional representation.

An Introduction to Variational Methods for Graphical Models

- Computer ScienceMachine Learning
- 2004

This paper presents a tutorial introduction to the use of variational methods for inference and learning in graphical models (Bayesian networks and Markov random fields), and describes a general framework for generating variational transformations based on convex duality.

Probabilistic topic models

- Computer ScienceCommun. ACM
- 2010

Surveying a suite of algorithms that offer a solution to managing large document archives suggests they are well-suited to handle large amounts of data.

Probabilistic Graphical Models - Principles and Techniques

- Computer Science
- 2009

The framework of probabilistic graphical models, presented in this book, provides a general approach for causal reasoning and decision making under uncertainty, allowing interpretable models to be constructed and then manipulated by reasoning algorithms.

Variational inference in nonconjugate models

- Computer ScienceJ. Mach. Learn. Res.
- 2013

These methods allow for easily derived variational algorithms with a wide class of nonconjugate models; they extend and unify some of the existing algorithms that have been derived for specific models; and they work well on real-world data sets.