Residual Sum of Squares Function
A Residual Sum of Squares Function is a metric based on the summation of the squares of residuals.
- AKA: Sum of Squares Error.
- Example(s):
- LLSE[math]\displaystyle{ (\lt 1,6\gt , \lt 2,5\gt , \lt 3,7\gt , \lt 4,10\gt ) \Rightarrow y=1.4x+3.5 }[/math], with residuals [math]\displaystyle{ +1.1, -1.3, -0.7, +0.9 }[/math] and sum of squares error of [math]\displaystyle{ 4.2 }[/math].
- …
- Counter-Example(s):
- See: Least Squares Estimation, Regression Tree Splitting Criterion.
References
2011a
- (Wikipedia, 2011) ⇒ http://en.wikipedia.org/wiki/Residual_sum_of_squares
- In statistics, the residual sum of squares (RSS) is the sum of squares of residuals. It is also known as the sum of squared residuals (SSR) or the sum of squared errors of prediction (SSE). It is a measure of the discrepancy between the data and an estimation model. A small RSS indicates a tight fit of the model to the data. In general, total sum of squares = explained sum of squares + residual sum of squares. For a proof of this in the multivariate ordinary least squares (OLS) case, see partitioning in the general OLS model.
In a model with a single explanatory variable, RSS is given by [math]\displaystyle{ RSS = \sum_{i=1}^n (y_i - f(x_i))^2, }[/math] where yi is the i th value of the variable to be predicted, xi is the i th value of the explanatory variable, and [math]\displaystyle{ f(x_i) }[/math] is the predicted value of yi.
In a standard linear simple regression model, [math]\displaystyle{ y_i = a+bx_i+\varepsilon_i\, }[/math], where a and b are coefficients, y and x are the regressand and the regressor, respectively, and ε is the error term. The sum of squares of residuals is the sum of squares of estimates of εi; that is [math]\displaystyle{ RSS = \sum_{i=1}^n (y_i - (\alpha + \beta x_i))^2, }[/math] where [math]\displaystyle{ \alpha }[/math] is the estimated value of the constant term [math]\displaystyle{ a }[/math] and [math]\displaystyle{ \beta }[/math] is the estimated value of the slope coefficient b.
- In statistics, the residual sum of squares (RSS) is the sum of squares of residuals. It is also known as the sum of squared residuals (SSR) or the sum of squared errors of prediction (SSE). It is a measure of the discrepancy between the data and an estimation model. A small RSS indicates a tight fit of the model to the data. In general, total sum of squares = explained sum of squares + residual sum of squares. For a proof of this in the multivariate ordinary least squares (OLS) case, see partitioning in the general OLS model.
2011b
- (Wikipedia, 2011) ⇒ http://en.wikipedia.org/wiki/Least_squares#Solving_the_least_squares_problem
- The minimum of the sum of squares is found by setting the gradient to zero. Since the model contains m parameters there are m gradient equations. [math]\displaystyle{ \frac{\partial S}{\partial \beta_j}=2\sum_i r_i\frac{\partial r_i}{\partial \beta_j}=0,\ j=1,\ldots,m }[/math] and since [math]\displaystyle{ r_i=y_i-f(x_i,\boldsymbol \beta)\, }[/math] the gradient equations become [math]\displaystyle{ -2\sum_i \frac{\partial f(x_i,\boldsymbol \beta)}{\partial \beta_j} r_i=0,\ j=1,\ldots,m }[/math].
The gradient equations apply to all least squares problems. Each particular problem requires particular expressions for the model and its partial derivatives.
- The minimum of the sum of squares is found by setting the gradient to zero. Since the model contains m parameters there are m gradient equations. [math]\displaystyle{ \frac{\partial S}{\partial \beta_j}=2\sum_i r_i\frac{\partial r_i}{\partial \beta_j}=0,\ j=1,\ldots,m }[/math] and since [math]\displaystyle{ r_i=y_i-f(x_i,\boldsymbol \beta)\, }[/math] the gradient equations become [math]\displaystyle{ -2\sum_i \frac{\partial f(x_i,\boldsymbol \beta)}{\partial \beta_j} r_i=0,\ j=1,\ldots,m }[/math].