# Law of total expectation

The proposition in probability theory known as the law of total expectation,[1] the law of iterated expectations[2] (LIE), Adam's law,[3] the tower rule,[4] and the smoothing theorem,[5] among other names, states that if $\displaystyle{ X }$ is a random variable whose expected value $\displaystyle{ \operatorname{E}(X) }$ is defined, and $\displaystyle{ Y }$ is any random variable on the same probability space, then

$\displaystyle{ \operatorname{E} (X) = \operatorname{E} ( \operatorname{E} ( X \mid Y)), }$

i.e., the expected value of the conditional expected value of $\displaystyle{ X }$ given $\displaystyle{ Y }$ is the same as the expected value of $\displaystyle{ X }$.

One special case states that if $\displaystyle{ {\left\{A_i\right\}}_i }$ is a finite or countable partition of the sample space, then

$\displaystyle{ \operatorname{E} (X) = \sum_i{\operatorname{E}(X \mid A_i) \operatorname{P}(A_i)}. }$

Note: The conditional expected value E(X | Z) is a random variable whose value depend on the value of Z. Note that the conditional expected value of X given the event Z = z is a function of z. If we write E(X | Z = z) = g(z) then the random variable E(X | Z) is g(Z). Similar comments apply to the conditional covariance.

## Example

Suppose that only two factories supply light bulbs to the market. Factory $\displaystyle{ X }$'s bulbs work for an average of 5000 hours, whereas factory $\displaystyle{ Y }$'s bulbs work for an average of 4000 hours. It is known that factory $\displaystyle{ X }$ supplies 60% of the total bulbs available. What is the expected length of time that a purchased bulb will work for?

Applying the law of total expectation, we have:

\displaystyle{ \begin{align} \operatorname{E} (L) &= \operatorname{E}(L \mid X) \operatorname{P}(X)+\operatorname{E}(L \mid Y) \operatorname{P}(Y) \\[3pt] &= 5000(0.6)+4000(0.4)\\[2pt] &=4600 \end{align} }

where

• $\displaystyle{ \operatorname{E} (L) }$ is the expected life of the bulb;
• $\displaystyle{ \operatorname{P}(X)={6 \over 10} }$ is the probability that the purchased bulb was manufactured by factory $\displaystyle{ X }$;
• $\displaystyle{ \operatorname{P}(Y)={4 \over 10} }$ is the probability that the purchased bulb was manufactured by factory $\displaystyle{ Y }$;
• $\displaystyle{ \operatorname{E}(L \mid X)=5000 }$ is the expected lifetime of a bulb manufactured by $\displaystyle{ X }$;
• $\displaystyle{ \operatorname{E}(L \mid Y)=4000 }$ is the expected lifetime of a bulb manufactured by $\displaystyle{ Y }$.

Thus each purchased light bulb has an expected lifetime of 4600 hours.

## Proof in the finite and countable cases

Let the random variables $\displaystyle{ X }$ and $\displaystyle{ Y }$, defined on the same probability space, assume a finite or countably infinite set of finite values. Assume that $\displaystyle{ \operatorname{E}[X] }$ is defined, i.e. $\displaystyle{ \min (\operatorname{E}[X_+], \operatorname{E}[X_-]) \lt \infty }$. If $\displaystyle{ \{A_i\} }$ is a partition of the probability space $\displaystyle{ \Omega }$, then

$\displaystyle{ \operatorname{E} (X) = \sum_i{\operatorname{E}(X \mid A_i) \operatorname{P}(A_i)}. }$

Proof.

\displaystyle{ \begin{align} \operatorname{E} \left( \operatorname{E} (X \mid Y) \right) &= \operatorname{E} \Bigg[ \sum_x x \cdot \operatorname{P}(X=x \mid Y) \Bigg] \\[6pt] &=\sum_y \Bigg[ \sum_x x \cdot \operatorname{P}(X=x \mid Y=y) \Bigg] \cdot \operatorname{P}(Y=y) \\[6pt] &=\sum_y \sum_x x \cdot \operatorname{P}(X=x, Y=y). \end{align} }

If the series is finite, then we can switch the summations around, and the previous expression will become

\displaystyle{ \begin{align} \sum_x \sum_y x \cdot \operatorname{P}(X=x, Y=y)&=\sum_x x\sum_y \operatorname{P}(X=x, Y=y)\\[6pt] &=\sum_x x \cdot \operatorname{P}(X=x)\\[6pt] &=\operatorname{E}(X). \end{align} }

If, on the other hand, the series is infinite, then its convergence cannot be conditional, due to the assumption that $\displaystyle{ \min (\operatorname{E}[X_+], \operatorname{E}[X_-] ) \lt \infty. }$ The series converges absolutely if both $\displaystyle{ \operatorname{E}[X_+] }$ and $\displaystyle{ \operatorname{E}[X_-] }$ are finite, and diverges to an infinity when either $\displaystyle{ \operatorname{E}[X_+] }$ or $\displaystyle{ \operatorname{E}[X_-] }$ is infinite. In both scenarios, the above summations may be exchanged without affecting the sum.

## Proof in the general case

Let $\displaystyle{ (\Omega,\mathcal{F},\operatorname{P}) }$ be a probability space on which two sub σ-algebras $\displaystyle{ \mathcal{G}_1 \subseteq \mathcal{G}_2 \subseteq \mathcal{F} }$ are defined. For a random variable $\displaystyle{ X }$ on such a space, the smoothing law states that if $\displaystyle{ \operatorname{E}[X] }$ is defined, i.e. $\displaystyle{ \min(\operatorname{E}[X_+], \operatorname{E}[X_-])\lt \infty }$, then

$\displaystyle{ \operatorname{E}[ \operatorname{E}[X \mid \mathcal{G}_2] \mid \mathcal{G}_1] = \operatorname{E}[X \mid \mathcal{G}_1]\quad\text{(a.s.)}. }$

Proof. Since a conditional expectation is a Radon–Nikodym derivative, verifying the following two properties establishes the smoothing law:

• $\displaystyle{ \operatorname{E}[ \operatorname{E}[X \mid \mathcal{G}_2] \mid \mathcal{G}_1] \mbox{ is } \mathcal{G}_1 }$-measurable
• $\displaystyle{ \int_{G_1} \operatorname{E}[ \operatorname{E}[X \mid \mathcal{G}_2] \mid \mathcal{G}_1] d\operatorname{P} = \int_{G_1} X d\operatorname{P}, }$ for all $\displaystyle{ G_1 \in \mathcal{G}_1. }$

The first of these properties holds by definition of the conditional expectation. To prove the second one,

\displaystyle{ \begin{align} \min\left(\int_{G_1}X_+\, d\operatorname{P}, \int_{G_1}X_-\, d\operatorname{P}\right) &\leq \min\left(\int_\Omega X_+\, d\operatorname{P}, \int_\Omega X_-\, d\operatorname{P}\right)\\[4pt] &=\min(\operatorname{E}[X_+], \operatorname{E}[X_-]) \lt \infty, \end{align} }

so the integral $\displaystyle{ \textstyle \int_{G_1}X\, d\operatorname{P} }$ is defined (not equal $\displaystyle{ \infty - \infty }$).

The second property thus holds since $\displaystyle{ G_1 \in \mathcal{G}_1 \subseteq \mathcal{G}_2 }$ implies

$\displaystyle{ \int_{G_1} \operatorname{E}[ \operatorname{E}[X \mid \mathcal{G}_2] \mid \mathcal{G}_1] d\operatorname{P} = \int_{G_1} \operatorname{E}[X \mid \mathcal{G}_2] d\operatorname{P} = \int_{G_1} X d\operatorname{P}. }$

Corollary. In the special case when $\displaystyle{ \mathcal{G}_1 = \{\empty,\Omega \} }$ and $\displaystyle{ \mathcal{G}_2 = \sigma(Y) }$, the smoothing law reduces to

$\displaystyle{ \operatorname{E}[ \operatorname{E}[X \mid Y]] = \operatorname{E}[X]. }$

Alternative proof for $\displaystyle{ \operatorname{E}[ \operatorname{E}[X \mid Y]] = \operatorname{E}[X]. }$

This is a simple consequence of the measure-theoretic definition of conditional expectation. By definition, $\displaystyle{ \operatorname{E}[X \mid Y] := \operatorname{E}[X \mid \sigma(Y)] }$ is a $\displaystyle{ \sigma(Y) }$-measurable random variable that satisfies

$\displaystyle{ \int_{A}\operatorname{E}[X \mid Y] d\operatorname{P} = \int_{A} X d\operatorname{P}, }$

for every measurable set $\displaystyle{ A \in \sigma(Y) }$. Taking $\displaystyle{ A = \Omega }$ proves the claim.

## Proof of partition formula

\displaystyle{ \begin{align} \sum\limits_i\operatorname{E}(X\mid A_i)\operatorname{P}(A_i) &=\sum\limits_i\int\limits_\Omega X(\omega)\operatorname{P}(d\omega\mid A_i)\cdot\operatorname{P}(A_i)\\ &=\sum\limits_i\int\limits_\Omega X(\omega)\operatorname{P}(d\omega\cap A_i)\\ &=\sum\limits_i\int\limits_\Omega X(\omega)I_{A_i}(\omega)\operatorname{P}(d\omega)\\ &=\sum\limits_i\operatorname{E}(XI_{A_i}), \end{align} }

where $\displaystyle{ I_{A_i} }$ is the indicator function of the set $\displaystyle{ A_i }$.

If the partition $\displaystyle{ {\{A_i\}}_{i=0}^n }$ is finite, then, by linearity, the previous expression becomes

$\displaystyle{ \operatorname{E}\left(\sum\limits_{i=0}^n XI_{A_i}\right)=\operatorname{E}(X), }$

and we are done.

If, however, the partition $\displaystyle{ {\{A_i\}}_{i=0}^\infty }$ is infinite, then we use the dominated convergence theorem to show that

$\displaystyle{ \operatorname{E}\left(\sum\limits_{i=0}^n XI_{A_i}\right)\to\operatorname{E}(X). }$

Indeed, for every $\displaystyle{ n\geq 0 }$,

$\displaystyle{ \left|\sum_{i=0}^n XI_{A_i}\right|\leq |X|I_{\mathop{\bigcup}\limits_{i=0}^n A_i}\leq |X|. }$

Since every element of the set $\displaystyle{ \Omega }$ falls into a specific partition $\displaystyle{ A_i }$, it is straightforward to verify that the sequence $\displaystyle{ {\left\{\sum_{i=0}^n XI_{A_i}\right\}}_{n=0}^\infty }$ converges pointwise to $\displaystyle{ X }$. By initial assumption, $\displaystyle{ \operatorname{E}|X|\lt \infty }$. Applying the dominated convergence theorem yields the desired result.

## References

1. Weiss, Neil A. (2005). A Course in Probability. Boston: Addison–Wesley. pp. 380–383. ISBN 0-321-18954-X.
2. Rhee, Chang-han (Sep 20, 2011). "Probability and Statistics".
3. Wolpert, Robert (November 18, 2010). "Conditional Expectation".