Radial Basis Kernel Function

A Radial Basis Kernel Function is a kernel function that is a radial basis function.

• AKA: RBF Kernel.
• Example(s):
  • a Gaussian RBF Kernel, for a Gaussian RBF.
  • …
• Counter-Example(s):
  • a Spectral-Mixture Kernel.
  • SVM-based Classifier.
  • Linear Discriminant Analysis.
• See: Positive-Definite Kernel, Distance Measure, Feature Space.

---

---

References

2016

• https://people.orie.cornell.edu/andrew/code/
  • QUOTE: Kernel Selection: The generalisation properties of a kernel method entirely depend on a kernel function. However, often one defaults to the RBF kernel, which can only discover very limited representations of data.

2014

• (Wikipedia, 2014) ⇒ http://en.wikipedia.org/wiki/radial_basis_function_kernel Retrieved:2014-9-22.
  • In machine learning, the (Gaussian) radial basis function kernel, or RBF kernel, is a popular kernel function used in support vector machine classification.

    The RBF kernel on two samples x and x', represented as feature vectors in some input space, is defined as  :[math]\displaystyle{ K(\mathbf{x}, \mathbf{x'}) = \exp\left(-\frac{||\mathbf{x} - \mathbf{x'}||_2^2}{2\sigma^2}\right) }[/math]

    [math]\displaystyle{ \textstyle||\mathbf{x} - \mathbf{x'}||_2^2 }[/math] may be recognized as the squared Euclidean distance between the two feature vectors. [math]\displaystyle{ \sigma }[/math] is a free parameter. An equivalent, but simpler, definition involves a parameter [math]\displaystyle{ \textstyle\gamma = -\tfrac{1}{2\sigma^2} }[/math]:  :[math]\displaystyle{ K(\mathbf{x}, \mathbf{x'}) = \exp(\gamma||\mathbf{x} - \mathbf{x'}||_2^2) }[/math]

    Since the value of the RBF kernel decreases with distance and ranges between zero (in the limit) and one (when ), it has a ready interpretation as a similarity measure.

    The feature space of the kernel has an infinite number of dimensions; for [math]\displaystyle{ \sigma = 1 }[/math], its expansion is:  :[math]\displaystyle{ \exp\left(-\frac{1}{2}||\mathbf{x} - \mathbf{x'}||_2^2\right) = \sum_{j=0}^\infty \frac{(\mathbf{x}^\top \mathbf{x'})^j}{j!} \exp\left(-\frac{1}{2}||\mathbf{x}||_2^2\right) \lt P\gt \exp\left(-\frac{1}{2}||\mathbf{x'}||_2^2\right) }[/math]

---