2015 StochasticDivergenceMinimizatio

• (Sato & Nakagawa, 2015) ⇒ Issei Sato, and Hiroshi Nakagawa. (2015). “Stochastic Divergence Minimization for Online Collapsed Variational Bayes Zero Inference of Latent Dirichlet Allocation.” In: Proceedings of the 21st ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD-2015). ISBN:978-1-4503-3664-2 doi:10.1145/2783258.2783355

Subject Headings:

Cited By

• http://scholar.google.com/scholar?q=%222015%22+Stochastic+Divergence+Minimization+for+Online+Collapsed+Variational+Bayes+Zero+Inference+of+Latent+Dirichlet+Allocation
• http://dl.acm.org/citation.cfm?id=2783258.2783355&preflayout=flat#citedby

Quotes

Author Keywords

• Collapsed variational bayes inference; latent dirichlet allocation; nonparametric statistics; online learning; stochastic optimization; topic modeling; variational bayes inference

Abstract

The collapsed variational Bayes zero (CVB0) inference is a variational inference improved by marginalizing out parameters, the same as with the collapsed Gibbs sampler. A drawback of the CVB0 inference is the memory requirements. A probability vector must be maintained for latent topics for every token in a corpus. When the total number of tokens is N and the number of topics is K, the CVB0 inference requires Ο (NK) memory. A stochastic approximation of the CVB0 (SCVB0) inference can reduce Ο (NK) to Ο (VK), where V denotes the vocabulary size. We reformulate the existing SCVB0 inference by using the stochastic divergence minimization algorithm, with which convergence can be analyzed in terms of Martingale convergence theory. We also reveal the property of the CVB0 inference in terms of the leave-one-out perplexity, which leads to the estimation algorithm of the Dirichlet distribution parameters. The predictive performance of the propose SCVB0 inference is better than that of the original SCVB0 inference in four datasets.

References

;

	Author	volume	Date Value	title	type	journal	titleUrl	doi	note	year
2015 StochasticDivergenceMinimizatio	Hiroshi Nakagawa Issei Sato			Stochastic Divergence Minimization for Online Collapsed Variational Bayes Zero Inference of Latent Dirichlet Allocation				10.1145/2783258.2783355		2015