1999 ExploitingGenerativeModelsinDis

(Jaakkola & Haussler, 1999) ⇒ Tommi S. Jaakkola, and David Haussler. (1999). “Exploiting Generative Models in Discriminative Classifiers.” In: Proceedings of the 1998 conference on Advances in Neural Information Processing Systems II. ISBN:0-262-11245-0

Subject Headings: Kernel Regression.

Notes

Cited By

Quotes

Abstract

Generative probability models such as hidden Markov models provide a principled way of treating missing information and dealing with variable length sequences. On the other hand, discriminative methods such as support vector machines enable us to construct flexible decision boundaries and often result in classification performance superior to that of the model based approaches. An ideal classifier should combine these two complementary approaches. In this paper, we develop a natural xvay of achieving this combination by deriving kernel functions for use in discriminative methods such as support vector machines from generative probability models. We provide a theoretical justification for this combination as well as demonstrate a substantial improvement in the classification performance in the context of DNA and protein sequence analysis.

1 Introduction

Speech, vision, text and biosequence data can be difficult to deal with in the context of simple statistical classification problems. Because the examples to be classified are often sequences or arrays of variable size that may have been distorted in particular ways, it is common to estimate a generative model for such data, and then use Bayes rule to obtain a classifier from this model. However. many discriminative methods, which directly estimate a posterior probability for a class label (as in Gaussian process classifiers [5]) or a discriminant function for the class label (as in support vector machines [6]) have in other areas proven to be superior to generative models for classification problems. The problem is that there has been no systematic way to extract features or metric relations between examples for use with discriminative methods in the context of difficult data types such as those listed above. Here we propose a general method for extracting these discriminatory features using a generative model. V?"ile the features xve propose are generally applicable, they are most naturally suited to kernel methods.

…

References

;

	Author	volume	Date Value	title	type	journal	titleUrl	doi	note	year
1999 ExploitingGenerativeModelsinDis	David Haussler Tommi S. Jaakkola			Exploiting Generative Models in Discriminative Classifiers						1999