Nonparametric Model Learning Algorithm

A nonparametric model learning algorithm is a model learning algorithm/statistical modeling algorithm that makes few assumptions about underlying probability distributions.

• AKA: Nonparametric Statistical Procedure.
• Context:
  • It can (often) be implemented by a Nonparametric Model Learning System to solve a Nonparametric Model Learning Task.
• Example(s):
  • Nonparametric Regression Algorithm.
  • Nonparametric Classification Algorithm.
  • Boosting Algorithm.
  • Principal Components Analysis Algorithm.
  • Nonparametric Bayes Algorithm.
  • FSF Algorithm.
  • EM Algorithm.
  • …
• Counter-Example(s):
  • a Parametric Model Learning Algorithm/Parametric Statistical Modeling Algorithm.
• See: Gaussian Process Algorithm, Distribution-Free Statistic, Distribution Function.

---

---

References

• http://en.wikipedia.org/wiki/Non-parametric_statistics

2009

• (Lafferty & Wasserman, 2009) ⇒ John D. Lafferty, and Larry Wasserman. (2009). “Statistical Machine Learning - Course: 10-702." Spring 2009, Carnegie Mellon Institute.
  • Nonparametric methods: Nonparametric Regression and Density Estimation, Nonparametric Classification, Boosting, Clustering and Dimension Reduction, PCA, Manifold Methods, Principal Curves, Spectral Methods, The Bootstrap and Subsampling, Nonparametric Bayes.

• (Ghahramani, 2009) ⇒ Zoubin Ghahramani. (2009). http://learning.eng.cam.ac.uk/zoubin/nonparam.html
  • QUOTE: Non-parametric models are very flexible statistical models in which the complexity of the model grows with the amount of observed data. While traditional parametric models make strong assumptions about how the data was generated, non-parametric models try to make weaker assumptions and let the data "speak for itself". Many non-parametric models can be seen as infinite limits of finite parametric models, and an important family of non-parametric models are derived from Dirichlet processes. See also Gaussian Processes.

2004

• (Zhao & Yu, 2004) ⇒ Peng Zhao, and Bin Yu. (2004). “Boosted Lasso." Tech Report, Statistics Department, U. C. Berkeley.
  • QUOTE: … FSF exists as a compromise since, like Boosting, it is a nonparametric learning algorithm that works with different loss functions and large numbers of base ...

1999

• (Hollander & Wolfe) ⇒ Myles Hollander, Douglas A. Wolfe. (1999). “Nonparametric Statistical Methods, 2nd Edition." Wiley. ISBN:0471190454
  • QUOTE: Roughly speaking, a nonparametric procedure is a statistical procedure that has certain desirable properties that hold under relatively mild assumptions regarding the underlying populations from which the data are obtained. That rapid and continuous development on nonparametric statistical procedures over that past six decades is due to the following advantages enjoyed by nonparametric techniques: ...
    • The term nonparametric, introduced in Section 1.1, is imprecise. The related term distribution-free has precise meaning. …

1998

• (Vijayakumar & Schaal, 1998) ⇒ Sethu Vijayakumar, and Stefan Schaal. (1998). “Local Adaptive Subspace Regression.” In: Neural Processing Letters, 7(3). doi:10.1023/A:1009696221209
  • QUOTE: … Based on this, we developed a nonparametric learning algorithm which is targeted to make use of such locally low dimensional distributions. ...

---