Uniformly most powerful test

Short description: Theoretically optimal hypothesis test

In statistical hypothesis testing, a uniformly most powerful (UMP) test is a hypothesis test which has the greatest power $1-\beta$ among all possible tests of a given size α. For example, according to the Neyman–Pearson lemma, the likelihood-ratio test is UMP for testing simple (point) hypotheses.

Setting

Let $X$ denote a random vector (corresponding to the measurements), taken from a parametrized family of probability density functions or probability mass functions $f_{\theta }(x),$ which depends on the unknown deterministic parameter $\theta \in \mathrm{\Theta }.$ The parameter space $\mathrm{\Theta }$ is partitioned into two disjoint sets ${\mathrm{\Theta }}_0$ and ${\mathrm{\Theta }}_1.$ Let $H_0$ denote the hypothesis that $\theta \in {\mathrm{\Theta }}_0,$ and let $H_1$ denote the hypothesis that $\theta \in {\mathrm{\Theta }}_1.$ The binary test of hypotheses is performed using a test function $\phi (x)$ with a reject region $R$ (a subset of measurement space).

$\phi (x)=\{\begin{array}{ll}1& \text{ if }x\in R\\ 0& \text{ if }x\in R^{\mathsf{C}}\end{array}$

meaning that $H_1$ is in force if the measurement $X\in R$ and that $H_0$ is in force if the measurement $X\in R^{\mathsf{C}}.$ Note that $R\cup R^c$ is a disjoint covering of the measurement space.

Formal definition

A test function $\phi (x)$ is UMP of size $\alpha$ if for any other test function ${\phi }^{\prime }(x)$ satisfying

$\underset{\theta \in {\mathrm{\Theta }}_0}{\sup }\mathrm{E}[{\phi }^{\prime }(X)|\theta ]={\alpha }^{\prime }\le \alpha =\underset{\theta \in {\mathrm{\Theta }}_0}{\sup }\mathrm{E}[\phi (X)|\theta ]$

we have

$\mathrm{\forall }\theta \in {\mathrm{\Theta }}_1,\mathrm{E}[{\phi }^{\prime }(X)|\theta ]=1-{\beta }^{\prime }(\theta )\le 1-\beta (\theta )=\mathrm{E}[\phi (X)|\theta ].$

The Karlin–Rubin theorem

The Karlin–Rubin theorem (named for Samuel Karlin and Herman Rubin) can be regarded as an extension of the Neyman–Pearson lemma for composite hypotheses. Consider a scalar measurement having a probability density function parameterized by a scalar parameter θ, and define the likelihood ratio $l(x)=f_{{\theta }_1}(x)/f_{{\theta }_0}(x)$. If $l(x)$ is monotone non-decreasing, in $x$, for any pair ${\theta }_1\ge {\theta }_0$ (meaning that the greater $x$ is, the more likely $H_1$ is), then the threshold test:

$\phi (x)=\{\begin{array}{ll}1& \text{if }x>x_0\\ 0& \text{if }x<x_0\end{array}$

where $x_0$ is chosen such that ${\mathrm{E}}_{{\theta }_0}\phi (X)=\alpha$

is the UMP test of size α for testing $H_0:\theta \le {\theta }_0\text{ vs. }{H}_1:\theta >{\theta }_0.$

Note that exactly the same test is also UMP for testing $H_0:\theta ={\theta }_0\text{ vs. }{H}_1:\theta >{\theta }_0.$

Important case: exponential family

Although the Karlin-Rubin theorem may seem weak because of its restriction to scalar parameter and scalar measurement, it turns out that there exist a host of problems for which the theorem holds. In particular, the one-dimensional exponential family of probability density functions or probability mass functions with

$f_{\theta }(x)=g(\theta )h(x)\exp (\eta (\theta )T(x))$

has a monotone non-decreasing likelihood ratio in the sufficient statistic $T(x)$, provided that $\eta (\theta )$ is non-decreasing.

Example

Let $X=(X_0,\dots ,X_{M-1})$ denote i.i.d. normally distributed $N$-dimensional random vectors with mean $\theta m$ and covariance matrix $R$. We then have

$\begin{array}{rl}{f}_{\theta }(X)=& (2\pi {)}^{-MN/2}|R{|}^{-M/2}\exp \{-\frac{1}{2}\sum _{n=0}^{M-1}(X_n-\theta m{)}^T{R}^{-1}(X_n-\theta m)\}\\ =& (2\pi {)}^{-MN/2}|R{|}^{-M/2}\exp \{-\frac{1}{2}\sum _{n=0}^{M-1}\left({\theta }^2{m}^T{R}^{-1}m\right)\}\\ & \exp \{-\frac{1}{2}\sum _{n=0}^{M-1}{X}_n^T{R}^{-1}{X}_n\}\exp \left\{\theta m^T{R}^{-1}\sum _{n=0}^{M-1}{X}_n\right\}\end{array}$

which is exactly in the form of the exponential family shown in the previous section, with the sufficient statistic being

$T(X)=m^T{R}^{-1}\sum _{n=0}^{M-1}{X}_n.$

Thus, we conclude that the test

$\phi (T)=\{\begin{array}{ll}1& T>t_0\\ 0& T<t_0\end{array}{\mathrm{E}}_{{\theta }_0}\phi (T)=\alpha$

is the UMP test of size $\alpha$ for testing $H_0:\theta ⩽{\theta }_0$ vs. $H_1:\theta >{\theta }_0$

Further discussion

In general, UMP tests do not exist for vector parameters or for two-sided tests (a test in which one hypothesis lies on both sides of the alternative). The reason is that in these situations, the most powerful test of a given size for one possible value of the parameter (e.g. for ${\theta }_1$ where ${\theta }_1>{\theta }_0$) is different from the most powerful test of the same size for a different value of the parameter (e.g. for ${\theta }_2$ where ${\theta }_2<{\theta }_0$). As a result, no test is uniformly most powerful in these situations.

Further reading

• Ferguson, T. S. (1967). Mathematical Statistics: A decision theoretic approach. New York: Academic Press. 
• Mood, A. M.; Graybill, F. A.; Boes, D. C. (1974). Introduction to the theory of statistics (3rd ed.). New York: McGraw-Hill. 
• L. L. Scharf, Statistical Signal Processing, Addison-Wesley, 1991, section 4.7.