2013 ScalableInferenceinMaxMarginTop

• (Zhu et al., 2013) ⇒ Jun Zhu, Xun Zheng, Li Zhou, and Bo Zhang. (2013). “Scalable Inference in Max-margin Topic Models.” In: Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ISBN:978-1-4503-2174-7 doi:10.1145/2487575.2487658

Subject Headings:

Cited By

• http://scholar.google.com/scholar?q=%222013%22+Scalable+Inference+in+Max-margin+Topic+Models
• http://dl.acm.org/citation.cfm?id=2487575.2487658&preflayout=flat#citedby

Quotes

Author Keywords

• Inference; large-scale systems; max-margin learning; probabilistic algorithms; statistical computing; topic models

Abstract

Topic models have played a pivotal role in analyzing large collections of complex data. Besides discovering latent semantics, supervised topic models (STMs) can make predictions on unseen test data. By marrying with advanced learning techniques, the predictive strengths of STMs have been dramatically enhanced, such as max-margin supervised topic models, state-of-the-art methods that integrate max-margin learning with topic models. Though powerful, max-margin STMs have a hard non-smooth learning problem. Existing algorithms rely on solving multiple latent SVM subproblems in an EM-type procedure, which can be too slow to be applicable to large-scale categorization tasks.

In this paper, we present a highly scalable approach to building max-margin supervised topic models. Our approach builds on three key innovations: 1) a new formulation of Gibbs max-margin supervised topic models for both multi-class and multi-label classification; 2) a simple ``augment-and-collapse " Gibbs sampling algorithm without making restricting assumptions on the posterior distributions; 3) an efficient parallel implementation that can easily tackle data sets with hundreds of categories and millions of documents. Furthermore, our algorithm does not need to solve SVM subproblems. Though performing the two tasks of topic discovery and [[learning predictive models jointly]], which significantly improves the classification performance, our methods have comparable scalability as the state-of-the-art parallel algorithms for the standard LDA topic models which perform the single task of topic discovery only. Finally, an open-source implementation is also provided at: http://www.ml-thu.net/~jun / medlda.

References

;

	Author	volume	Date Value	title	type	journal	titleUrl	doi	note	year
2013 ScalableInferenceinMaxMarginTop	Jun Zhu Bo Zhang Xun Zheng Li Zhou			Scalable Inference in Max-margin Topic Models				10.1145/2487575.2487658		2013