Maximum A Posteriori Estimation

MAP estimation chooses the mode of the posterior distribution. By Bayes’ theorem,

$$p(\theta \mid D)=\frac{p(D\mid \theta )p(\theta )}{p(D)},$$

Here $D$ is the observed dataset, $\theta$ is the unknown parameter, $p(D\mid \theta )$ is the likelihood, and $p(\theta )$ is the prior. The posterior $p(\theta \mid D)$ is the updated distribution over parameter values after seeing the data.

so the denominator can be ignored for optimization:

$${\hat{\theta }}_{\mathrm{MAP}}=\arg \underset{\theta }{\max }p(\theta \mid D)=\arg \underset{\theta }{\max }p(D\mid \theta )p(\theta ).$$

The evidence term $p(D)$ does not depend on $\theta$, so it changes the scale of the posterior but not the parameter value that maximizes it.

It is maximum likelihood plus a prior. In many ML objectives, the log-prior acts like regularization: a Gaussian prior on weights yields an $L_2$ penalty.

Worked scenario

A coin is tossed 10 times and lands heads 7 times. With a flat $\mathrm{Beta}(1,1)$ prior, the posterior mode is the observed head rate, $0.7$, because the prior contributes no pull toward either side. With a symmetric $\mathrm{Beta}(2,2)$ prior, the prior behaves like one prior head and one prior tail for the posterior mode, giving $(7+1)/(10+2)=0.6667$.

A much stronger symmetric prior, $\mathrm{Beta}(10,10)$, behaves like nine prior heads and nine prior tails for the mode, giving $(7+9)/(10+18)=0.5714$. The likelihood still favors heads, but the prior encodes a strong expectation that the coin is near fair, so the MAP estimate moves toward $0.5$ rather than staying at the sample proportion.

Caveats

MAP hides posterior uncertainty and can be sensitive to parameterization: a mode can move under nonlinear transformations. Bayesian statistics often reports posterior means, medians, intervals, or decisions rather than only the posterior mode.

References

• Maximum a posteriori estimation
• SciPy beta distribution

Nav

Section — Probability and Statistics

← Maximum Likelihood Bayesian Statistics →