Ordinary Least-Squares Regression System

An Ordinary Least-Squares Regression System is a least-squares regression system that is an Linear Regression System which implements an ordinary least-squares regression algorithm to solve an ordinary least-squares regression task.

• AKA: OLS System.
• Context:
  • It can range from being a Linear Ordinary Least-Squares Regression System to being a Non-LInear Ordinary Least-Squares Regression System.
• Example(s):
  • sklearn.linear_model.LinearRegression [1].
  • statsmodels.api.OLS [2]
• Counter-Example(s):
  • a Ridge Regression System.
  • a Nonlinear Least-Squares Regression System.
• See: Regression Analysis System, Linear Regression Function, scikit-learn Advanced Data Science Library.

---

---

References

2017a

• (Scikit Learn, 2017) ⇒ http://scikit-learn.org/stable/modules/linear_model.html Retrieved: 2017-30-07.
  • QUOTE: The following are a set of methods intended for regression in which the target value is expected to be a linear combination of the input variables. In mathematical notion, if [math]\displaystyle{ \hat{y} }[/math] is the predicted value.

    [math]\displaystyle{ \hat{y}(w, x) = w_0 + w_1 x_1 + \cdots + w_p x_p }[/math]

    Across the module, we designate the vector [math]\displaystyle{ w = (w_1,\cdots, w_p) }[/math] as coef_ and [math]\displaystyle{ w_0 }[/math] as intercept_.

2017b

• (StatsModels, 2017) ⇒ http://www.statsmodels.org/stable/regression.html Retrieved:2017-09-10
  • QUOTE: The statistical model is assumed to be

    [math]\displaystyle{ Y = X\beta + \mu }[/math],

    where [math]\displaystyle{ \mu\sim N\left(0,\Sigma\right) }[/math]

    Depending on the properties of [math]\displaystyle{ \Sigma }[/math], we have currently four classes available:

    • GLS : generalized least squares for arbitrary covariance [math]\displaystyle{ \Sigma }[/math]
    • OLS : ordinary least squares for i.i.d. errors [math]\displaystyle{ \Sigma=\textbf{I} }[/math]
    • WLS : weighted least squares for heteroskedastic errors [math]\displaystyle{ \text{diag}\left (\Sigma\right) }[/math]
    • GLSAR : feasible generalized least squares with autocorrelated AR(p) errors [math]\displaystyle{ \Sigma=\Sigma\left(\rho\right) }[/math]

    All regression models define the same methods and follow the same structure, and can be used in a similar fashion. Some of them contain additional model specific methods and attributes.

---