Taylor expansions for the moments of functions of random variables

In probability theory, it is possible to approximate the moments of a function f of a random variable X using Taylor expansions, provided that f is sufficiently differentiable and that the moments of X are finite.

First moment

[math]\displaystyle{ \begin{align} \operatorname{E}\left[f(X)\right] & {} = \operatorname{E}\left[f\left(\mu_X + \left(X - \mu_X\right)\right)\right] \\ & {} \approx \operatorname{E}\left[f(\mu_X) + f'(\mu_X)\left(X-\mu_X\right) + \frac{1}{2}f''(\mu_X) \left(X - \mu_X\right)^2 \right]. \end{align} }[/math]

Since [math]\displaystyle{ E[X-\mu_X]=0, }[/math] the second term disappears. Also [math]\displaystyle{ E[(X-\mu_X)^2] }[/math] is [math]\displaystyle{ \sigma_X^2 }[/math]. Therefore,

[math]\displaystyle{ \operatorname{E}\left[f(X)\right]\approx f(\mu_X) +\frac{f''(\mu_X)}{2}\sigma_X^2 }[/math]

where [math]\displaystyle{ \mu_X }[/math] and [math]\displaystyle{ \sigma^2_X }[/math] are the mean and variance of X respectively.

It is possible to generalize this to functions of more than one variable using multivariate Taylor expansions. For example,

[math]\displaystyle{ \operatorname{E}\left[\frac{X}{Y}\right]\approx\frac{\operatorname{E}\left[X\right]}{\operatorname{E}\left[Y\right]} -\frac{\operatorname{cov}\left[X,Y\right]}{\operatorname{E}\left[Y\right]^2}+\frac{\operatorname{E}\left[X\right]}{\operatorname{E}\left[Y\right]^3}\operatorname{var}\left[Y\right] }[/math]

Second moment

Similarly,

[math]\displaystyle{ \operatorname{var}\left[f(X)\right]\approx \left(f'(\operatorname{E}\left[X\right])\right)^2\operatorname{var}\left[X\right] = \left(f'(\mu_X)\right)^2\sigma^2_X }[/math]

The above is using a first order approximation unlike for the method used in estimating the first moment. It will be a poor approximation in cases where [math]\displaystyle{ f(X) }[/math] is highly non-linear. This is a special case of the delta method. For example,

[math]\displaystyle{ \operatorname{var}\left[\frac{X}{Y}\right]\approx\frac{\operatorname{var}\left[X\right]}{\operatorname{E}\left[Y\right]^2}-\frac{2\operatorname{E}\left[X\right]}{\operatorname{E}\left[Y\right]^3}\operatorname{cov}\left[X,Y\right]+\frac{\operatorname{E}\left[X\right]^2}{\operatorname{E}\left[Y\right]^4}\operatorname{var}\left[Y\right]. }[/math]

The second order approximation is:

[math]\displaystyle{ \operatorname{var}\left[f(X)\right]\approx \left(f'(\operatorname{E}\left[X\right])\right)^2\operatorname{var}\left[X\right] + \frac{\left(f''(\operatorname{E}\left[X\right])\right)^2}{2}\left(\operatorname{var}\left[X\right]\right)^2 = \left(f'(\mu_X)\right)^2\sigma^2_X + \frac{1}{2}\left(f''(\mu_X)\right)^2\sigma_X^4 }[/math]

See also

• Propagation of uncertainty
• WKB approximation
• Delta method

Further reading

• Wolter, Kirk M. (1985). "Taylor Series Methods". Introduction to Variance Estimation. New York: Springer. pp. 221–247. ISBN 0-387-96119-4. https://books.google.com/books?id=EadxTw0t2dMC&pg=PA221.