A Revised Inference for Correlated Topic Model

Abstract

In this paper, we provide a revised inference for correlated topic model (CTM) [3]. CTM is proposed by Blei et al. for modeling correlations among latent topics more expressively than latent Dirichlet allocation (LDA) [2] and has been attracting attention of researchers. However, we have found that the variational inference of the original paper is unstable due to almost-singularity of the covariance matrix when the number of topics is large. This means that we may be reluctant to use CTM for analyzing a large document set, which may cover a rich diversity of topics. Therefore, we revise the inference and improve its quality. First, we modify the formula for updating the covariance matrix in a manner that enables us to recover the original inference by adjusting a parameter. Second, we regularize posterior parameters for reducing a side effect caused by modifying the formula. While our method is based on a heuristic intuition, an experiment conducted on large document sets showed that it worked effectively in terms of perplexity.

DOI: 10.1007/978-3-642-39068-5_54

Extracted Key Phrases

3 Figures and Tables

Cite this paper

@inproceedings{Masada2013ARI, title={A Revised Inference for Correlated Topic Model}, author={Tomonari Masada and Atsuhiro Takasu}, booktitle={ISNN}, year={2013} }