2007 ThreeNewGraphicalModelsforStati

• (Mnih & Hinton, 2007) ⇒ Andriy Mnih, and Geoffrey Hinton. (2007). “Three New Graphical Models for Statistical Language Modelling.” In: Proceedings of the 24th International Conference on Machine learning. ISBN:978-1-59593-793-3 doi:10.1145/1273496.1273577

Subject Headings:

Cited By

• http://scholar.google.com/scholar?q=%222007%22+Three+New+Graphical+Models+for+Statistical+Language+Modelling
• http://dl.acm.org/citation.cfm?id=1273496.1273577&preflayout=flat#citedby

Quotes

Abstract

The supremacy of [n-gram models in statistical language modelling has recently been challenged by parametric models that use distributed representations to counteract the difficulties caused by data sparsity. We propose three new probabilistic language models that define the distribution of the next word in a sequence given several preceding words by using distributed representations of those words. We show how real-valued distributed representations for words can be learned at the same time as learning a large set of stochastic binary hidden features that are used to predict the distributed representation of the next word from previous distributed representations. Adding connections from the previous states of the binary hidden features improves performance as does adding direct connections between the real-valued distributed representations. One of our models significantly outperforms the very best n-gram models.

References

;

	Author	volume	Date Value	title	type	journal	titleUrl	doi	note	year
2007 ThreeNewGraphicalModelsforStati	Geoffrey E. Hinton Andriy Mnih			Three New Graphical Models for Statistical Language Modelling				10.1145/1273496.1273577		2007