Maximum a Posteriori Estimate
A Maximum a Posteriori Estimate is a point estimate that ...
- AKA: MAP.
- Context:
- It can be produced by a Maximum a Posteriori Estimation System (to solve a MAP estimation task).
- It can be a Maximum Likelihood Estimate when no prior distribution is used.
- …
- Counter-Example(s):
- See: argmax, Bayesian Inference.
References
2015
- (Wikipedia, 2015) ⇒ http://en.wikipedia.org/wiki/Maximum_a_posteriori_estimation Retrieved:2015-6-15.
- In Bayesian statistics, a maximum a posteriori probability (MAP) estimate is a mode of the posterior distribution. The MAP can be used to obtain a point estimate of an unobserved quantity on the basis of empirical data. It is closely related to Fisher's method of maximum likelihood (ML), but employs an augmented optimization objective which incorporates a prior distribution over the quantity one wants to estimate. MAP estimation can therefore be seen as a regularization of ML estimation.
2011
- (Wikipedia, 2011) http://en.wikipedia.org/wiki/Maximum_a_posteriori
- In Bayesian statistics, a maximum a posteriori probability (MAP) estimate is a mode of the posterior distribution. The MAP can be used to obtain a point estimate of an unobserved quantity on the basis of empirical data. It is closely related to Fisher's method of maximum likelihood (ML), but employs an augmented optimization objective which incorporates a prior distribution over the quantity one wants to estimate. MAP estimation can therefore be seen as a regularization of ML estimation.
Assume that we want to estimate an unobserved population parameter [math]\displaystyle{ \theta }[/math] on the basis of observations [math]\displaystyle{ x }[/math]. Let [math]\displaystyle{ f }[/math] be the sampling distribution of [math]\displaystyle{ x }[/math], so that [math]\displaystyle{ f(x|\theta) }[/math] is the probability of [math]\displaystyle{ x }[/math] when the underlying population parameter is [math]\displaystyle{ \theta }[/math]. Then the function [math]\displaystyle{ \theta \mapsto f(x | \theta) \! }[/math] is known as the likelihood function and the estimate [math]\displaystyle{ \displaystyle \hat{\theta}_{\mathrm{ML}}(x) = \mathop{\mbox{arg max }}_{\theta}\ f(x \vert \theta) \! }[/math] is the maximum likelihood estimate of [math]\displaystyle{ \theta }[/math]. Now assume that a prior distribution [math]\displaystyle{ g }[/math] over [math]\displaystyle{ \theta }[/math] exists. This allows us to treat [math]\displaystyle{ \theta }[/math] as a random variable as in Bayesian statistics. Then the posterior distribution of [math]\displaystyle{ \theta }[/math] is as follows: [math]\displaystyle{ \theta \mapsto f(\theta | x) = \frac{f(x | \theta) \, g(\theta)}{\displaystyle \int_{\theta' \in \Theta} f(x | \theta') \, g(\theta') \, d\theta'} \! }[/math] where [math]\displaystyle{ g }[/math] is density function of [math]\displaystyle{ \theta }[/math], [math]\displaystyle{ \Theta }[/math] is the domain of [math]\displaystyle{ g }[/math]. This is a straightforward application of Bayes' theorem. The method of maximum a posteriori estimation then estimates [math]\displaystyle{ \theta }[/math] as the mode of the posterior distribution of this random variable: [math]\displaystyle{ \displaystyle \hat{\theta}_{\mathrm{MAP}}(x) = \mathop{\mbox{arg max }}_{\theta} \ \frac{f(x | \theta) \, g(\theta)} {\displaystyle\int_{\Theta} f(x | \theta') \, g(\theta') \, d\theta'} = \mathop{\mbox{arg max }}_{\theta} \ f(x | \theta) \, g(\theta). \! }[/math]
The denominator of the posterior distribution (so-called partition function) does not depend on [math]\displaystyle{ \theta }[/math] and therefore plays no role in the optimization. Observe that the MAP estimate of [math]\displaystyle{ \theta }[/math] coincides with the ML estimate when the prior [math]\displaystyle{ g }[/math] is uniform (that is, a constant function). The MAP estimate is a limit of Bayes estimators under a sequence of 0-1 loss functions, but generally not a Bayes estimator per se, unless [math]\displaystyle{ \theta }[/math] is discrete.
- In Bayesian statistics, a maximum a posteriori probability (MAP) estimate is a mode of the posterior distribution. The MAP can be used to obtain a point estimate of an unobserved quantity on the basis of empirical data. It is closely related to Fisher's method of maximum likelihood (ML), but employs an augmented optimization objective which incorporates a prior distribution over the quantity one wants to estimate. MAP estimation can therefore be seen as a regularization of ML estimation.
2007
- (Awate, 2007) ⇒ Suyash P. Awate. (2007). “Adaptive Nonparametric Markov Models and Information-Theoretic Methods for Image Restoration and Segmentation.” Ph.D. thesis, The University of Utah.
- QUOTE: Sometimes we have a priori information about the physical process whose parameters we want to estimate. Such information can come either from the correct scientific knowledge of the physical process or from previous empirical evidence. We can encode such prior information in terms of a PDF on the parameter to be estimated. Essentially, we treat the parameter [math]\displaystyle{ \theta }[/math] as the value of an RV. The associated probabilities [math]\displaystyle{ P(\theta) }[/math] are called the prior probabilities. We refer to the inference based on such priors as Bayesian inference. Bayes' theorem shows the way for incorporating prior information in the estimation process: [math]\displaystyle{ \displaystyle P (θ \vert {\bf x}) = \frac { P ({\bf x} \vert θ) P (θ) } { P ({\bf x}) } (35) }[/math] The term on the left hand side of the equation is called the posterior. On the right hand side, the numerator is the product of the likelihood term and the prior term. The denominator serves as a normalization term so that the posterior PDF integrates to unity. Thus, Bayesian inference produces the maximum a posteriori (MAP) estimate :[math]\displaystyle{ \displaystyle \mathop{\mbox{argmax }}_{θ} P (θ \vert {\bf x}) = \mathop{\mbox{argmax }}_{θ} P ({\bf x} \vert θ) P (θ). (36) }[/math]