Online Learning for Latent Dirichlet Allocation

Abstract

We develop an online variational Bayes (VB) algorithm for Latent Dirichlet Allocation (LDA). Online LDA is based on online stochastic optimization with a natural gradient step, which we show converges to a local optimum of the VB objective function. It can handily analyze massive document collections, including those arriving in a stream. We study the performance of online LDA in several ways, including by fitting a 100-topic topic model to 3.3M articles from Wikipedia in a single pass. We demonstrate that online LDA finds topic models as good or better than those found with batch VB, and in a fraction of the time.

Extracted Key Phrases

2 Figures and Tables

05010015020102011201220132014201520162017
Citations per Year

743 Citations

Semantic Scholar estimates that this publication has 743 citations based on the available data.

See our FAQ for additional information.

Cite this paper

@inproceedings{Hoffman2010OnlineLF, title={Online Learning for Latent Dirichlet Allocation}, author={Matthew D. Hoffman and David M. Blei and Francis R. Bach}, booktitle={NIPS}, year={2010} }