Quadratic Discriminant Analysis Algorithm

A Quadratic Discriminant Analysis Algorithm is a Discriminant Analysis Algorithm that ...

• AKA: Quadratic Classifier.
  • …
• Counter-Example(s):
  • Linear Discriminant Analysis.
• See: Likelihood-Ratio Test, Normal Distribution, Covariance.

---

---

References

2015

• (Wikipedia, 2015) ⇒ http://en.wikipedia.org/wiki/Quadratic_classifier#Quadratic_discriminant_analysis Retrieved:2015-10-23.
  • Quadratic discriminant analysis (QDA) is closely related to linear discriminant analysis (LDA), where it is assumed that the measurements from each class are normally distributed. Unlike LDA however, in QDA there is no assumption that the covariance of each of the classes is identical. When the normality assumption is true, the best possible test for the hypothesis that a given measurement is from a given class is the likelihood ratio test. Suppose there are only two groups, (so [math]\displaystyle{ y \in \{0,1 \} }[/math] ), and the means of each class are defined to be [math]\displaystyle{ \mu_{y=0},\mu_{y=1} }[/math] and the covariances are defined as [math]\displaystyle{ \Sigma_{y=0}, \Sigma_{y=1} }[/math] . Then the likelihood ratio will be given by

    :Likelihood ratio = [math]\displaystyle{ \frac{ \sqrt{2 \pi |\Sigma_{y=1}|}^{-1} \exp \left( -\frac{1}{2}(x-\mu_{y=1})^T \Sigma_{y=1}^{-1} (x-\mu_{y=1}) \right) }{ \sqrt{2 \pi |\Sigma_{y=0}|}^{-1} \exp \left( -\frac{1}{2}(x-\mu_{y=0})^T \Sigma_{y=0}^{-1} (x-\mu_{y=0}) \right)} \lt t }[/math]

    for some threshold t. After some rearrangement, it can be shown that the resulting separating surface between the classes is a quadratic. The sample estimates of the mean vector and variance-covariance matrices will substitute the population quantities in this formula.

---