Bayesian Regression System
A Bayesian Regression System is a Regression System that implements an Bayesian Regression Algorithm to solve a Bayesian Regression Task.
- AKA: Bayesian Regressor.
- Context:
- It ranges from being a ARD Regression System to being a Bayesian Ridge Regression System.
- It ranges from being a Linear Bayesian Regression System to being a Nonlinear Bayesian Regression System.
- …
- Example(s):
- Counter-Example(s):
- See: Cross-Validation Task, Regression Analysis Task, Bayesian Inference, Bayes Theorem.
References
2017
- (Scikit Learn, 2017) ⇒ http://scikit-learn.org/stable/modules/linear_model.html#bayesian-regression Retrieved:2017-09-17
- QUOTE: Bayesian regression techniques can be used to include regularization parameters in the estimation procedure: the regularization parameter is not set in a hard sense but tuned to the data at hand.
This can be done by introducing uninformative priors over the hyperparameters of the model. The [math]\displaystyle{ \ell_{2} }[/math] regularization used in Ridge Regression is equivalent to finding a maximum a posteriori estimation under a Gaussian prior over the parameters w with precision [math]\displaystyle{ \lambda^{-1} }[/math]. Instead of setting
lambda
manually, it is possible to treat it as a random variable to be estimated from the data.To obtain a fully probabilistic model, the output [math]\displaystyle{ y }[/math] is assumed to be Gaussian distributed around [math]\displaystyle{ X w }[/math]:
[math]\displaystyle{ p(y|X,w,\alpha) = \mathcal{N}(y|X w,\alpha) }[/math]
Alpha is again treated as a random variable that is to be estimated from the data.
The advantages of Bayesian Regression are:
- It adapts to the data at hand.
- It can be used to include regularization parameters in the estimation procedure.
- QUOTE: Bayesian regression techniques can be used to include regularization parameters in the estimation procedure: the regularization parameter is not set in a hard sense but tuned to the data at hand.
- The disadvantages of Bayesian regression include:
- Inference of the model can be time consuming.
- The disadvantages of Bayesian regression include: