Generative Classification Function

A Generative Classification Function is a generative model that is a classification function.

• AKA: Generative Classification Model.
• Context:
  • It can be produced by a Generative Classification Algorithm.
  • …
• Example(s):
  • A Hidden Markov Model.
  • An Linear Discriminant Analysis Model.
  • …
• Counter-Example(s):
  • a Generative Regression Function.
  • a Discriminative Classification Model.
• See: Naive Bayes, Bayes Theorem.

---

---

References

2004

• (Bouchard & Triggs, 2004) ⇒ Guillaume Bouchard, and Bill Triggs. (2004). “The Trade-off Between Generative and Discriminative Classifiers.” In: Proceedings of COMPSTAT 2004.
  • QUOTE: .. More generally, it is known that generative classifiers have a smaller variance than … if the overall goal is to find the classification rule with the smallest error rate, this depends only on the conditional density [math]\displaystyle{ p(y \vert x) }[/math]. “Discriminative" methods directly model the conditional distribution, without assuming anything about the input distribution p(x). Well known generative-discriminative pairs include Linear Discriminant Analysis (LDA) vs. Linear logistic regression and naive Bayes vs. Generalized Additive Models (GAM). Many authors have already studied these models e.g. [5,6]. Under the assumption that the underlying distributions are Gaussian with equal covariances, it is known that LDA requires less data than its discriminative counterpart, linear logistic regression [3]. More generally, it is known that generative classifiers have a smaller variance than …

---