Bussgang theorem

In mathematics, the Bussgang theorem is a theorem of stochastic analysis. The theorem states that the cross-correlation between a Gaussian signal before and after it has passed through a nonlinear operation are equal to the signals auto-correlation up to a constant. It was first published by Julian J. Bussgang in 1952 while he was at the Massachusetts Institute of Technology.

Statement

Let [math]\displaystyle{ \left\{X(t)\right\} }[/math] be a zero-mean stationary Gaussian random process and [math]\displaystyle{ \left \{ Y(t) \right\} = g(X(t)) }[/math] where [math]\displaystyle{ g(\cdot) }[/math] is a nonlinear amplitude distortion.

If [math]\displaystyle{ R_X(\tau) }[/math] is the autocorrelation function of [math]\displaystyle{ \left\{ X(t) \right\} }[/math], then the cross-correlation function of [math]\displaystyle{ \left\{ X(t) \right\} }[/math] and [math]\displaystyle{ \left\{ Y(t) \right\} }[/math] is

[math]\displaystyle{ R_{XY}(\tau) = CR_X(\tau), }[/math]

where [math]\displaystyle{ C }[/math] is a constant that depends only on [math]\displaystyle{ g(\cdot) }[/math].

It can be further shown that

[math]\displaystyle{ C = \frac{1}{\sigma^3\sqrt{2\pi}}\int_{-\infty}^\infty ug(u)e^{-\frac{u^2}{2\sigma^2}} \, du. }[/math]

Derivation for One-bit Quantization

It is a property of the two-dimensional normal distribution that the joint density of [math]\displaystyle{ y_1 }[/math] and [math]\displaystyle{ y_2 }[/math] depends only on their covariance and is given explicitly by the expression

[math]\displaystyle{ p(y_1,y_2) = \frac{1}{2 \pi \sqrt{1-\rho^2}} e^{-\frac{y_1^2 + y_2^2 - 2 \rho y_1 y_2}{2(1-\rho^2)}} }[/math]

where [math]\displaystyle{ y_1 }[/math] and [math]\displaystyle{ y_2 }[/math] are standard Gaussian random variables with correlation [math]\displaystyle{ \phi_{y_1y_2}=\rho }[/math].

Assume that [math]\displaystyle{ r_2 = Q(y_2) }[/math], the correlation between [math]\displaystyle{ y_1 }[/math] and [math]\displaystyle{ r_2 }[/math] is,

[math]\displaystyle{ \phi_{y_1r_2} = \frac{1}{2 \pi \sqrt{1-\rho^2}} \int_{-\infty}^{\infty} \int_{-\infty}^{\infty} y_1 Q(y_2) e^{-\frac{y_1^2 + y_2^2 - 2 \rho y_1 y_2}{2(1-\rho^2)}} \, dy_1 dy_2 }[/math].

Since

[math]\displaystyle{ \int_{-\infty}^{\infty} y_1 e^{-\frac{1}{2(1-\rho^2)} y_1^2 + \frac{\rho y_2}{1-\rho^2} y_1 } \, dy_1 = \rho \sqrt{2 \pi (1-\rho^2)} y_2 e^{ \frac{\rho^2 y_2^2}{2(1-\rho^2)} } }[/math],

the correlation [math]\displaystyle{ \phi_{y_1 r_2} }[/math] may be simplified as

[math]\displaystyle{ \phi_{y_1 r_2} = \frac{\rho}{\sqrt{2 \pi}} \int_{-\infty}^{\infty} y_2 Q(y_2) e^{-\frac{y_2^2}{2}} \, dy_2 }[/math].

The integral above is seen to depend only on the distortion characteristic [math]\displaystyle{ Q() }[/math] and is independent of [math]\displaystyle{ \rho }[/math].

Remembering that [math]\displaystyle{ \rho=\phi_{y_1 y_2} }[/math], we observe that for a given distortion characteristic [math]\displaystyle{ Q() }[/math], the ratio [math]\displaystyle{ \frac{\phi_{y_1 r_2}}{\phi_{y_1 y_2}} }[/math] is [math]\displaystyle{ K_Q=\frac{1}{\sqrt{2\pi}} \int_{-\infty}^{\infty} y_2 Q(y_2) e^{-\frac{y_2^2}{2}} \, dy_2 }[/math].

Therefore, the correlation can be rewritten in the form

[math]\displaystyle{ \phi_{y_1 r_2} = K_Q \phi_{y_1 y_2} }[/math].

The above equation is the mathematical expression of the stated "Bussgang‘s theorem".

If [math]\displaystyle{ Q(x) = \text{sign}(x) }[/math], or called one-bit quantization, then [math]\displaystyle{ K_Q= \frac{2}{\sqrt{2\pi}} \int_{0}^{\infty} y_2 e^{-\frac{y_2^2}{2}} \, dy_2 = \sqrt{\frac{2}{\pi}} }[/math].

Arcsine law

If the two random variables are both distorted, i.e., [math]\displaystyle{ r_1 = Q(y_1), r_2 = Q(y_2) }[/math], the correlation of [math]\displaystyle{ r_1 }[/math] and [math]\displaystyle{ r_2 }[/math] is

[math]\displaystyle{ \phi_{r_1 r_2}=\int_{-\infty}^{\infty}\int_{-\infty}^{\infty} Q(y_1) Q(y_2) p(y_1, y_2) \, dy_1 dy_2 }[/math].

When [math]\displaystyle{ Q(x) = \text{sign}(x) }[/math], the expression becomes,

[math]\displaystyle{ \phi_{r_1 r_2}=\frac{1}{2\pi \sqrt{1-\rho^2}} \left[ \int_{0}^{\infty} \int_{0}^{\infty} e^{-\alpha} \, dy_1 dy_2 + \int_{-\infty}^{0} \int_{-\infty}^{0} e^{-\alpha} \, dy_1 dy_2 - \int_{0}^{\infty} \int_{-\infty}^{0} e^{-\alpha} \, dy_1 dy_2 - \int_{-\infty}^{0} \int_{0}^{\infty} e^{-\alpha} \, dy_1 dy_2 \right] }[/math]

where [math]\displaystyle{ \alpha = \frac{y_1^2 + y_2^2 - 2\rho y_1 y_2}{2 (1-\rho^2)} }[/math].

Noticing that

[math]\displaystyle{ \int_{-\infty}^{\infty} \int_{-\infty}^{\infty} p(y_1,y_2) \, dy_1 dy_2 = \frac{1}{2\pi \sqrt{1-\rho^2}} \left[ \int_{0}^{\infty} \int_{0}^{\infty} e^{-\alpha} \, dy_1 dy_2 + \int_{-\infty}^{0} \int_{-\infty}^{0} e^{-\alpha} \, dy_1 dy_2 + \int_{0}^{\infty} \int_{-\infty}^{0} e^{-\alpha} \, dy_1 dy_2 + \int_{-\infty}^{0} \int_{0}^{\infty} e^{-\alpha} \, dy_1 dy_2 \right]=1 }[/math],

and [math]\displaystyle{ \int_{0}^{\infty} \int_{0}^{\infty} e^{-\alpha} \, dy_1 dy_2 = \int_{-\infty}^{0} \int_{-\infty}^{0} e^{-\alpha} \, dy_1 dy_2 }[/math], [math]\displaystyle{ \int_{0}^{\infty} \int_{-\infty}^{0} e^{-\alpha} \, dy_1 dy_2 = \int_{-\infty}^{0} \int_{0}^{\infty} e^{-\alpha} \, dy_1 dy_2 }[/math],

we can simplify the expression of [math]\displaystyle{ \phi_{r_1r_2} }[/math] as

[math]\displaystyle{ \phi_{r_1 r_2}=\frac{4}{2\pi \sqrt{1-\rho^2}} \int_{0}^{\infty} \int_{0}^{\infty} e^{-\alpha} \, dy_1 dy_2-1 }[/math]

Also, it is convenient to introduce the polar coordinate [math]\displaystyle{ y_1 = R \cos \theta, y_2 = R \sin \theta }[/math]. It is thus found that

[math]\displaystyle{ \phi_{r_1 r_2} =\frac{4}{2\pi \sqrt{1-\rho^2}} \int_{0}^{\pi/2} \int_{0}^{\infty} e^{-\frac{R^2 - 2R^2 \rho \cos \theta \sin \theta \ }{2(1-\rho^2)}} R \, dR d\theta-1=\frac{4}{2\pi \sqrt{1-\rho^2}} \int_{0}^{\pi/2} \int_{0}^{\infty} e^{-\frac{R^2 (1-\rho \sin 2\theta )}{2(1-\rho^2)}} R \, dR d\theta -1 }[/math].

Integration gives

[math]\displaystyle{ \phi_{r_1 r_2}=\frac{2\sqrt{1-\rho^2}}{\pi} \int_{0}^{\pi/2} \frac{d\theta}{1-\rho \sin 2\theta} - 1= - \frac{2}{\pi} \arctan \left( \frac{\rho-\tan\theta} {\sqrt{1-\rho^2}} \right) \Bigg|_{0}^{\pi/2} -1 =\frac{2}{\pi} \arcsin(\rho) }[/math]，

This is called "Arcsine law", which was first found by J. H. Van Vleck in 1943 and republished in 1966. The "Arcsine law" can also be proved in a simpler way by applying Price's Theorem.

The function [math]\displaystyle{ f(x)=\frac{2}{\pi} \arcsin x }[/math] can be approximated as [math]\displaystyle{ f(x) \approx \frac{2}{\pi} x }[/math] when [math]\displaystyle{ x }[/math] is small.

Price's Theorem

Given two jointly normal random variables [math]\displaystyle{ y_1 }[/math] and [math]\displaystyle{ y_2 }[/math] with joint probability function

[math]\displaystyle{ {\displaystyle p(y_{1},y_{2})={\frac {1}{2\pi {\sqrt {1-\rho ^{2}}}}}e^{-{\frac {y_{1}^{2}+y_{2}^{2}-2\rho y_{1}y_{2}}{2(1-\rho ^{2})}}}} }[/math],

we form the mean

[math]\displaystyle{ I(\rho)=E(g(y_1,y_2))=\int_{-\infty}^{+\infty} \int_{-\infty}^{+\infty} g(y_1, y_2) p(y_1, y_2) \, dy_1 dy_2 }[/math]

of some function [math]\displaystyle{ g(y_1,y_2) }[/math] of [math]\displaystyle{ (y_1, y_2) }[/math]. If [math]\displaystyle{ g(y_1, y_2) p(y_1, y_2) \rightarrow 0 }[/math] as [math]\displaystyle{ (y_1, y_2) \rightarrow 0 }[/math], then

[math]\displaystyle{ \frac{\partial^n I(\rho)}{\partial \rho^n}=\int_{-\infty}^{\infty} \int_{-\infty}^{\infty} \frac{\partial ^{2n} g(y_1, y_2)}{\partial y_1^n \partial y_2^n} p(y_1, y_2) \, dy_1 dy_2 =E \left(\frac{\partial ^{2n} g(y_1, y_2)}{\partial y_1^n \partial y_2^n} \right) }[/math].

Proof. The joint characteristic function of the random variables [math]\displaystyle{ y_1 }[/math] and [math]\displaystyle{ y_2 }[/math] is by definition the integral

[math]\displaystyle{ \Phi(\omega_1, \omega_2)=\int_{-\infty}^{\infty}\int_{-\infty}^{\infty} p(y_1, y_2) e^{j (\omega_1 y_1 + \omega_2 y_2 )} \, dy_1 dy_2 = \exp \left\{-\frac{\omega_1^2 + \omega_2^2 + 2\rho \omega_1 \omega_2}{2} \right\} }[/math].

From the two-dimensional inversion formula of Fourier transform, it follows that

[math]\displaystyle{ p(y_1, y_2) = \frac{1}{4 \pi^2} \int_{-\infty}^{\infty} \int_{-\infty}^{\infty} \Phi(\omega_1, \omega_2) e^{-j (\omega_1 y_1 + \omega_2 y_2)} \, d\omega_1 d\omega_2 =\frac{1}{4 \pi^2} \int_{-\infty}^{\infty} \int_{-\infty}^{\infty} \exp \left\{-\frac{\omega_1^2 + \omega_2^2 + 2\rho \omega_1 \omega_2}{2} \right\} e^{-j (\omega_1 y_1 + \omega_2 y_2)} \, d\omega_1 d\omega_2 }[/math].

Therefore, plugging the expression of [math]\displaystyle{ p(y_1, y_2) }[/math] into [math]\displaystyle{ I(\rho) }[/math], and differentiating with respect to [math]\displaystyle{ \rho }[/math], we obtain

[math]\displaystyle{ \begin{align} \frac{\partial^n I(\rho)}{\partial \rho^n} & = \int_{-\infty}^{\infty} \int_{-\infty}^{\infty} g(y_1, y_2) p(y_1, y_2) \, dy_1 dy_2 \\ & = \int_{-\infty}^{\infty} \int_{-\infty}^{\infty} g(y_1, y_2) \left(\frac{1}{4 \pi^2} \int_{-\infty}^{\infty} \int_{-\infty}^{\infty} \frac{\partial^ {n}\Phi(\omega_1, \omega_2)}{\partial \rho^n} e^{-j(\omega_1 y_1 + \omega_2 y_2)} \, d\omega_1 d\omega_2 \right) \, dy_1 dy_2 \\ & = \int_{-\infty}^{\infty} \int_{-\infty}^{\infty} g(y_1, y_2) \left(\frac{(-1)^n}{4 \pi^2} \int_{-\infty}^{\infty} \int_{-\infty}^{\infty} \omega_1^n \omega_2^n \Phi(\omega_1, \omega_2) e^{-j(\omega_1 y_1 + \omega_2 y_2)} \, d\omega_1 d\omega_2 \right) \, dy_1 dy_2 \\ & = \int_{-\infty}^{\infty} \int_{-\infty}^{\infty} g(y_1, y_2) \left(\frac{1}{4 \pi^2} \int_{-\infty}^{\infty} \int_{-\infty}^{\infty} \Phi(\omega_1, \omega_2) \frac{\partial^{2n} e^{-j(\omega_1 y_1 + \omega_2 y_2)}}{\partial y_1^n \partial y_2^n} \, d\omega_1 d\omega_2 \right) \, dy_1 dy_2 \\ & = \int_{-\infty}^{\infty} \int_{-\infty}^{\infty} g(y_1, y_2) \frac{\partial^{2n} p(y_1, y_2)}{\partial y_1^n \partial y_2^n} \, dy_1 dy_2 \\ \end{align} }[/math]

After repeated integration by parts and using the condition at [math]\displaystyle{ \infty }[/math], we obtain the Price's theorem.

[math]\displaystyle{ \begin{align} \frac{\partial^n I(\rho)}{\partial \rho^n} & = \int_{-\infty}^{\infty} \int_{-\infty}^{\infty} g(y_1, y_2) \frac{\partial^{2n} p(y_1, y_2)}{\partial y_1^n \partial y_2^n} \, dy_1 dy_2 \\ & = \int_{-\infty}^{\infty} \int_{-\infty}^{\infty} \frac{\partial^{2} g(y_1, y_2)}{\partial y_1 \partial y_2} \frac{\partial^{2n-2} p(y_1, y_2)}{\partial y_1^{n-1} \partial y_2^{n-1}} \, dy_1 dy_2 \\ &=\cdots \\ &=\int_{-\infty}^{\infty} \int_{-\infty}^{\infty} \frac{\partial ^{2n} g(y_1, y_2)}{\partial y_1^n \partial y_2^n} p(y_1, y_2) \, dy_1 dy_2 \end{align} }[/math]

Proof of Arcsine law by Price's Theorem

If [math]\displaystyle{ g(y_1, y_2) = \text{sign}(y_1) \text{sign} (y_2) }[/math], then [math]\displaystyle{ \frac{\partial^2 g(y_1, y_2)}{\partial y_1 \partial y_2} = 4 \delta(y_1) \delta(y_2) }[/math] where [math]\displaystyle{ \delta() }[/math] is the Dirac delta function.

Substituting into Price's Theorem, we obtain,

[math]\displaystyle{ \frac{\partial E(\text{sign} (y_1) \text{sign}(y_2))}{\partial \rho} = \frac{\partial I(\rho)}{\partial \rho}= \int_{-\infty}^{\infty} \int_{-\infty}^{\infty} 4 \delta(y_1) \delta(y_2) p(y_1, y_2) \, dy_1 dy_2=\frac{2}{\pi \sqrt{1-\rho^2}} }[/math].

When [math]\displaystyle{ \rho=0 }[/math], [math]\displaystyle{ I(\rho)=0 }[/math]. Thus

[math]\displaystyle{ E \left(\text{sign}(y_1) \text{sign}(y_2) \right) = I(\rho)=\frac{2}{\pi} \int_{0}^{\rho} \frac{1}{\sqrt{1-\rho^2}} \, d\rho=\frac{2}{\pi} \arcsin(\rho) }[/math],

which is Van Vleck's well-known result of "Arcsine law".

Application

This theorem implies that a simplified correlator can be designed. Instead of having to multiply two signals, the cross-correlation problem reduces to the gating of one signal with another.

Further reading

• E.W. Bai; V. Cerone; D. Regruto (2007) "Separable inputs for the identification of block-oriented nonlinear systems", Proceedings of the 2007 American Control Conference (New York City, July 11–13, 2007) 1548–1553