Partial Least Squares (PLS) Regression Algorithm

A Partial Least Squares (PLS) Regression Algorithm is a least-squares regression algorithm that can be applied by a partial least-squares regression system (to solve a partial least-squares regression task).

• Context:
  • It can be a Dimensionality Reduction Algorithm.
  • It can contruct predictive models when the factors are many and highly collinear. (Tobias, 1995).
  • It was developed in the 1960’s by Herman Wold as an econometric technique. (Tobias, 1995).
  • …
• Counter-Example(s):
  • Principal Component Regression.
• See: Linear Regression, Predicted Variable, Observable Variable, Matrix (Mathematics), Latent Variable, Covariance, Multicollinearity, PLS Path Modelling, Structural Equation Modeling, Least Squares.

---

---

References

2017

• (Wikipedia, 2017) ⇒ https://en.wikipedia.org/wiki/Partial_least_squares_regression Retrieved:2017-11-1.
  • Partial least squares regression (PLS regression) is a statistical method that bears some relation to principal components regression; instead of finding hyperplanes of maximum variance between the response and independent variables, it finds a linear regression model by projecting the predicted variables and the observable variables to a new space. Because both the X and Y data are projected to new spaces, the PLS family of methods are known as bilinear factor models. Partial least squares Discriminant Analysis (PLS-DA) is a variant used when the Y is categorical.

    PLS is used to find the fundamental relations between two matrices (X and Y), i.e. a latent variable approach to modeling the covariance structures in these two spaces. A PLS model will try to find the multidimensional direction in the X space that explains the maximum multidimensional variance direction in the Y space. PLS regression is particularly suited when the matrix of predictors has more variables than observations, and when there is multicollinearity among X values. By contrast, standard regression will fail in these cases (unless it is regularized).

    Partial least squares was introduced by the Swedish statistician Herman Wold, who then developed it with his son, Svante Wold. An alternative term for PLS (and more correct according to Svante Wold ) is projection to latent structures, but the term partial least squares is still dominant in many areas. Although the original applications were in the social sciences, PLS regression is today most widely used in chemometrics and related areas. It is also used in bioinformatics, sensometrics, neuroscience and anthropology.

ism for being an unreliable estimation and testing tool.

2008

• (Saigo et al., 2008) ⇒ Hiroto Saigo, Nicole Krämer, and Koji Tsuda. (2008). “Partial Least Squares Regression for Graph Mining.” In: Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD-2008). doi:10.1145/1401890.1401961

2008b

• (Upton & Cook, 2008) ⇒ Graham Upton, and Ian Cook. (2008). “A Dictionary of Statistics, 2nd edition revised." Oxford University Press. ISBN:0199541450
  • QUOTE: Partial Least Squares (PLS): A method for handling correlated explanatory variables in the context of [[multiple regression. In PLS the first stage is to determine k uncorrelated variables that are linear combinations of the explanatory variables. The combinations are chosen for their predictive ability. Principal components regression analysis uses a different technique to achieve the same objective.

2006

• (Wold, 2006) ⇒ H. Wold. Partial Least Squares. doi:10.1002/0471667196.ess1914.pub2

1995

• (Tobias, 1995) ⇒ Randall D. Tobias. (1995). “An Introduction to Partial Least Squares Regression.

---