View source for Binomial sum variance inequality

The '''binomial sum variance inequality''' states that the variance of the sum of [[Binomial distribution|binomially distributed]] [[Random variable|random variable]]s will always be less than or equal to the variance of a binomial variable with the same [[Binomial distribution#Mean and variance|''n'' and ''p'']] parameters. In [[Probability theory|probability theory]] and [[Statistics|statistics]], the [[Binomial distribution#Sums of binomials|sum]] of independent binomial random variables is itself a binomial random variable if all the component variables share the same [[Binomial distribution#Mean and variance|success probability]]. If success probabilities differ, the probability distribution of the sum is not binomial.<ref>{{cite journal |last1=Butler |first1=Ken |last2=Stephens |first2=Michael |year=1993 |title=The distribution of a sum of binomial random variables |journal=Technical Report No. 467 |publisher=Department of Statistics, Stanford University |url=https://apps.dtic.mil/sti/pdfs/ADA266969.pdf |archive-url=https://web.archive.org/web/20210411183835/https://apps.dtic.mil/sti/pdfs/ADA266969.pdf |url-status=live |archive-date=April 11, 2021 }}</ref> The lack of uniformity in success probabilities across independent trials leads to a smaller variance.<ref>Nedelman, J and Wallenius, T., 1986. Bernoulli trials, Poisson trials, surprising variances, and Jensen’s Inequality. The American Statistician, 40(4):286–289.</ref><ref>Feller, W. 1968. An introduction to probability theory and its applications (Vol. 1, 3rd ed.). New York: John Wiley.</ref><ref>Johnson, N. L. and Kotz, S. 1969. Discrete distributions. New York: John Wiley</ref><ref>Kendall, M. and Stuart, A. 1977. The advanced theory of statistics. New York: Macmillan.</ref><ref name="DreznerFarnum1993">{{cite journal |last1=Drezner |first1=Zvi |last2=Farnum |first2=Nicholas |title=A generalized binomial distribution |journal=Communications in Statistics - Theory and Methods |volume=22 |issue=11 |year=1993 |pages=3051–3063 |issn=0361-0926 |doi=10.1080/03610929308831202}}</ref> and is a special case of a more general theorem involving the [[Expected value|expected value]] of convex functions.<ref>Hoeffding, W. 1956. On the distribution of the number of successes in independent trials. Annals of Mathematical Statistics (27):713–721.</ref> In some statistical applications, the standard binomial variance estimator can be used even if the component probabilities differ, though with a variance estimate that has an upward [[Bias (statistics)|bias]].
==Inequality statement==
Consider the sum, ''Z'', of two independent binomial random variables, ''X'' ~ B(''m''<sub>0</sub>, ''p''<sub>0</sub>) and ''Y'' ~ B(''m''<sub>1</sub>, ''p''<sub>1</sub>), where ''Z'' = ''X'' + ''Y''. Then, the variance of ''Z'' is less than or equal to its variance under the assumption that ''p''<sub>0</sub> = ''p''<sub>1</sub>, that is, if ''Z'' had a binomial distribution.<ref>{{cite journal |last1=Millstein |first1=J. |last2=Volfson |first2=D. |year=2013 |title=Computationally efficient permutation-based confidence interval estimation for tail-area FDR |journal=Frontiers in Genetics |volume=4 |issue=179 |pages=1–11 |pmc=3775454 |doi=10.3389/fgene.2013.00179 |pmid=24062767|doi-access=free }}</ref> Symbolically, <math>Var(Z) \leqslant E[Z] (1 - \tfrac{E[Z]}{m_0+m_1})</math>.

{{hidden begin|style=width:60%|ta1=center|border=1px #aaa solid|title=[Proof]}}
We wish to prove that 
:<math>Var(Z) \leqslant E[Z] (1 - \frac{E[Z]}{m_0+m_1})</math>
We will prove this inequality by finding an expression for Var(''Z'') and substituting it on the left-hand side, then showing that the inequality always holds.

If ''Z'' has a binomial distribution with parameters ''n'' and ''p'', then the [[Expected value|expected value]] of ''Z''  is given by [[Binomial distribution#Mean and variance|E[''Z''] = ''np'']] and the variance of ''Z'' is given by [[Binomial distribution#Mean and variance|Var[''Z''] = ''np''(1 – ''p'')]]. Letting ''n'' = ''m''<sub>0</sub> + ''m''<sub>1</sub> and substituting E[''Z''] for ''np'' gives 
:<math>Var(Z) = E[Z] (1 - \frac{E[Z]}{m_0+m_1})</math>
The random variables ''X'' and ''Y'' are independent, so the [[Variance#Sum of uncorrelated variables|variance of the sum is equal to the sum of the variances]], that is
:<math>Var(Z) = E[X] (1-\frac{E[X]}{m_0}) + E[Y] (1-\frac{E[Y]}{m_1})</math>

In order to prove the theorem, it is therefore sufficient to prove that
:<math>E[X](1 - \frac{E[X]}{m_0}) + E[Y](1 - \frac{E[Y]}{m_1}) \leqslant E[Z](1 - \frac{E[Z]}{m_0+m_1})</math>


Substituting E[''X''] + E[''Y''] for E[''Z''] gives
:<math>E[X](1 - \frac{E[X]}{m_0}) + E[Y](1 - \frac{E[Y]}{m_1}) \leqslant (E[X]+E[Y])(1 - \frac{E[X]+E[Y]}{m_0+m_1})</math>
Multiplying out the brackets and subtracting E[X] + E[Y] from both sides yields
:<math>- \frac{E[X]^2}{m_0} - \frac{E[Y]^2}{m_1} \leqslant - \frac{(E[X]+E[Y])^2}{m_0+m_1}</math>
Multiplying out the brackets yields
:<math>E[X] - \frac{E[X]^2}{m_0} + E[Y] - \frac{E[Y]^2}{m_1} \leqslant E[X] + E[Y] - \frac{(E[X]+E[Y])^2}{m_0+m_1}</math>
Subtracting E[X] and E[Y] from both sides and reversing the inequality gives
:<math>\frac{E[X]^2}{m_0} + \frac{E[Y]^2}{m_1} \geqslant \frac{(E[X]+E[Y])^2}{m_0+m_1}</math>
Expanding the right-hand side gives
:<math>\frac{E[X]^2}{m_0} + \frac{E[Y]^2}{m_1} \geqslant \frac{E[X]^2+2E[X]E[Y]+E[Y]^2}{m_0+m_1}</math>
Multiplying by <math>m_0 m_1 (m_0+m_1)</math> yields
:<math>(m_0m_1+{m_1}^2){E[X]^2}+ ({m_0}^2+m_0m_1){E[Y]^2} \geqslant m_0m_1({E[X]}^2+2E[X]E[Y]+{E[Y]]^2})</math>
Deducting the right-hand side gives the relation 
:<math>{m_1}^2{E[X]^2} -2m_0m_1E[X]E[Y] + {m_0}^2{E[Y]^2} \geqslant 0</math>
or equivalently
:<math>(m_1E[X] - m_0E[Y])^2 \geqslant 0</math>
The square of a real number is always greater than or equal to zero, so this is true for all independent binomial distributions that X and Y could take. This is sufficient to prove the theorem.
{{hidden end}}


Although this proof was developed for the sum of two variables, it is easily generalized to greater than two. Additionally, if the individual success probabilities are known, then the variance is known to take the form<ref name=DreznerFarnum1993 />

: <math> \operatorname{Var}(Z) = n \bar{p} (1 - \bar{p}) - ns^2,</math>

where <math> s^2 = \frac{1}{n}\sum_{i=1}^n (p_i-\bar{p})^2</math>. This expression also implies that the variance is always less than that of the binomial distribution with <math>p=\bar{p}</math>, because the standard expression for the variance is decreased by ''ns''<sup>2</sup>, a positive number.

==Applications==
The inequality can be useful in the context of [[Multiple comparisons problem|multiple testing]], where many [[Statistical hypothesis testing|statistical hypothesis tests]] are conducted within a particular study. Each test can be treated as a [[Bernoulli distribution|Bernoulli variable]] with a success probability ''p''. Consider the total number of positive tests as a random variable denoted by ''S''. This quantity is important in the estimation of [[False discovery rate|false discovery rates (FDR)]], which quantify uncertainty in the test results. If the [[Statistical hypothesis testing#the testing process|null hypothesis]] is true for some tests and the [[Statistical hypothesis testing#the testing process|alternative hypothesis]] is true for other tests, then success probabilities are likely to differ between these two groups. However, the variance inequality theorem states that if the tests are independent, the variance of ''S'' will be no greater than it would be under a binomial distribution.

==References==
{{reflist}}

[[Category:Probability theorems]]
[[Category:Theorems in statistics]]
[[Category:Statistical inequalities]]

{{Sourceattribution|Binomial sum variance inequality}}