Regularized Least-Squares Function Fitting Algorithm

A Regularized Least-Squares Function Fitting Algorithm is a least-squares function fitting algorithm that is a regularized optimization algorithm.

• AKA: RLSA.
• Context:
  • It can range from being a Regularized Least-Squares Estimation Algorithm to being a Regularized Least-Squares Ranking Algorithm to being a Regularized Least-Squares Classification Algorithm.
  • It can be applied by a Regularized Least Squares System (that can solve a Regularized Least Squares Task).
  • It can range from being an L1-Regularized Least Squares Algorithm/L1-Regularized LSA to being a L2-Regularized Least Squares Algorithm.
  • …
• Counter-Example(s):
  • a Non-Regularized Least Squares Algorithm.
  • a Regularized Logistic Regression Algorithm.
• See: Least-Squares.

---

---

References

2008

• (Airola, 2008) ⇒ Antti Airola, Sampo Pyysalo, Jari Björne, Tapio Pahikkala, Filip Ginter and Tapio Salakoski. (2008). “A Graph Kernel for Protein-Protein Interaction Extraction.” In: Proceedings of BioNLP 2008

2007

• (Caponnetto & De Vito, 2007) ⇒ Andrea Caponnetto, and Ernesto De Vito. (2007). “Optimal Rates for the Regularized Least-Squares Algorithm." Foundations of Computational Mathematics. doi:10.1007/s10208-006-0196-8
  • QUOTE: We develop a theoretical analysis of the performance of the regularized least-square algorithm on ... In this paper we investigate the estimation properties of the regularized least-squares (RLS) algorithm on ...
• (Kim, Koh et al., 2007) ⇒ Seung-Jean Kim, K. Koh, M. Lustig, Stephen Boyd, and Dimitry Gorinevsky. (2007). “An Interior-Point Method for Large-Scale l1-Regularized Least Squares.” In: IEEE Journal on Selected Topics in Signal Processing, 1(4). [doi>10.1109/JSTSP.2007.910971]

2005

• (De Vito et al., 2005) ⇒ E. De Vito, A. Caponnetto and L. Rosasco. (2005). “Model Selection for Regularized Least-Squares Algorithm in Learning Theory” doi:10.1007/s10208-004-0134-1

1998

• (Chen, Chng & Alkadhimi, 1996) ⇒ S. Chen, E. S. Chng, and Khalil Alkadhimi. (1996), "Regularized Orthogonal Least Squares Algorithm for Constructing Radial Basis Function Retworks.” In: International Journal of Control, 64(5). doi:10.1080/00207179608921659
  • ABSTRACT: The paper presents a regularized orthogonal least squares learning algorithm for radial basis function networks. The proposed algorithm combines the advantages of both the orthogonal forward regression and regularization methods to provide an efficient and powerful procedure for constructing parsimonious network models that generalize well. Examples of nonlinear modelling and prediction are used to demonstrate better generalization performance of this regularized orthogonal least squares algorithm over the unregularized one.

---