View source for Algebraic formula for the variance

In [[Probability theory|probability theory]] and [[Statistics|statistics]], there are several '''algebraic formulae for the variance''' available for deriving the [[Variance|variance]] of a [[Random variable|random variable]]. The usefulness of these depends on what is already known about the random variable; for example a random variable may be defined in terms of its [[Physics:Probability density function|probability density function]] or by construction from other random variables. The context here is that of deriving algebraic expressions for the theoretical variance of a random variable, in contrast to questions of estimating the variance of a [[Statistical population|population]] from sample data for which there are special considerations in implementing [[Algorithms for calculating variance|computational algorithms]].

==In terms of raw moments==
If the raw moments E(''X'') and E(''X'' <sup>2</sup>) of a [[Random variable|random variable]] ''X'' are known (where E(''X'') is the [[Expected value|expected value]] of ''X''), then Var(''X'') is given by

:<math>\operatorname{Var}(X) = \operatorname{E}(X^2) - [\operatorname{E}(X)]^2   .</math>

The result is called the König–[[Biography:Christiaan Huygens|Huygens]] formula in French-language literature<ref>In French: formule de Koenig–Huygens. See e.g. {{citation|title=Maths: prépas commerciales|first=Jean-Jacques|last=Martiano|publisher=Studyrama|year=2006|isbn=9782844728289|page=148}}</ref> and known as [[Biography:Jakob Steiner|Steiner]] translation theorem in Germany.<ref>In German: Verschiebungssatz von Steiner. See e.g. {{citation|title=Starthilfe Stochastik: Studium|first1=Gerd|last1=Christoph|first2=Horst|last2=Hackel|publisher=Springer|year=2013|isbn=9783322847997|page=50|url=https://books.google.com/books?id=Hiw25tqvScsC&pg=PA50}}.</ref>

There is a corresponding formula for use in estimation of the variance from sample data, that can be of use in hand calculations. This is a closely related identity  that is structured to create an [[Bias (statistics)|unbiased]] estimate of the population variance

:<math>
\hat{\sigma}^2 = \frac{1}{N-1}\sum_{i=1}^N(x_i-\bar{x})^2 = \frac{N}{N-1}\left(\frac{1}{N}\left(\sum_{i=1}^N x_i^2\right) - \bar{x}^2\right)
\equiv \frac{1}{N-1}\left(\left(\sum_{i=1}^N x_i^2\right) - N \left(\bar{x}\right)^2\right)   . 
</math>

However, use of these formulas can be unwise in practice when using floating point arithmetic with limited precision: subtracting two values having a similar magnitude can lead to [[Loss of significance|catastrophic cancellation]],<ref>Donald E. Knuth (1998). ''[[The Art of Computer Programming]]'', volume 2: ''Seminumerical Algorithms'', 3rd edn., p. 232. Boston: Addison-Wesley.</ref> and thus causing a loss of significance when <math>\operatorname{E}(X)^2 \gg \operatorname{Var}(X)</math>.<ref name=":0">{{Cite journal|last=Schubert|first=Erich|last2=Gertz|first2=Michael|date=2018-07-09|title=Numerically stable parallel computation of (co-)variance|url=http://dl.acm.org/citation.cfm?id=3221269.3223036|publisher=ACM|pages=10|doi=10.1145/3221269.3223036|isbn=9781450365055}}</ref>
There several other numerically stable [[Algorithms for calculating variance|algorithms for calculating variance]] for use with floating point numbers.<ref name=":0" />

===Proof===

The computational formula for the population variance follows in a straightforward manner from the [[Expected value#Linearity|linearity of expected values]] and the definition of variance:

:<math>
\begin{align}
\operatorname{Var}(X)&=\operatorname{E}\left[(X - \operatorname{E}(X))^2\right]\\
                     &=\operatorname{E}\left[X^2 - 2X\operatorname{E}(X) + [\operatorname{E}(X)]^2\right]\\
                     &=\operatorname{E}(X^2) - \operatorname{E}[2X\operatorname{E}(X)] + [\operatorname{E}(X)]^2\\
                     &=\operatorname{E}(X^2) - 2\operatorname{E}(X)\operatorname{E}(X) + [\operatorname{E}(X)]^2\\
                     &=\operatorname{E}(X^2) - 2[\operatorname{E}(X)]^2 + [\operatorname{E}(X)]^2\\
                     &=\operatorname{E}(X^2) - [\operatorname{E}(X)]^2
\end{align}
</math>

===Generalization to covariance===

This formula can be generalized for [[Covariance|covariance]], with two random variables X<sub>i</sub> and X<sub>j</sub>:

:<math>\operatorname{Cov}(X_i, X_j) = \operatorname{E}(X_iX_j) -\operatorname{E}(X_i)\operatorname{E}(X_j)</math>

as well as for the ''n'' by ''n'' [[Covariance matrix|covariance matrix]] of a random vector of length ''n'':

:<math> \operatorname{Var}(\mathbf{X}) = \operatorname{E}(\mathbf{X X^\top}) - \operatorname{E}(\mathbf{X})\operatorname{E}(\mathbf{X})^\top</math>

and for the ''n'' by ''m'' [[Cross-covariance|cross-covariance]] matrix ''between'' two random vectors of length ''n'' and ''m'':
:<math>
\operatorname{Cov}(\textbf{X},\textbf{Y})=
\operatorname{E}(\mathbf{X Y^\top}) - \operatorname{E}(\mathbf{X})\operatorname{E}(\mathbf{Y})^\top</math>

where expectations are taken element-wise and <math>\mathbf{X}=\{X_1,X_2,\ldots,X_n\}</math> and <math>\mathbf{Y}=\{Y_1,Y_2,\ldots,Y_m\}</math> are random vectors of respective lengths ''n'' and ''m''.

Note that this formula suffers from the same [[Loss of significance|loss of significance]] as the formula for variance if used for calculating estimates of the covariance, and alternative algorithms should be used instead.<ref name=":0" />

==See also==

*[[Standard deviation#Identities and mathematical properties|Standard deviation: Identities and mathematical properties]]

==References==

{{Reflist}}

[[Category:Statistical deviation and dispersion]]

{{Sourceattribution|Algebraic formula for the variance|1}}