Bussgang theorem

In mathematics, the Bussgang theorem is a theorem of stochastic analysis. The theorem states that the cross-correlation between a Gaussian signal before and after it has passed through a nonlinear operation are equal to the signals auto-correlation up to a constant. It was first published by Julian J. Bussgang in 1952 while he was at the Massachusetts Institute of Technology.^[1]

Statement

Let $\left\{X(t)\right\}$ be a zero-mean stationary Gaussian random process and $\left\{Y(t)\right\}=g(X(t))$ where $g(\cdot )$ is a nonlinear amplitude distortion.

If $R_{X}(\tau )$ is the autocorrelation function of $\left\{X(t)\right\}$ , then the cross-correlation function of $\left\{X(t)\right\}$ and $\left\{Y(t)\right\}$ is

R_{XY}(\tau )=CR_{X}(\tau ),

where $C$ is a constant that depends only on $g(\cdot )$ .

It can be further shown that

C={\frac {1}{\sigma ^{3}{\sqrt {2\pi }}}}\int _{-\infty }^{\infty }ug(u)e^{-{\frac {u^{2}}{2\sigma ^{2}}}}\,du.

Derivation for One-bit Quantization

It is a property of the two-dimensional normal distribution that the joint density of $y_{1}$ and $y_{2}$ depends only on their covariance and is given explicitly by the expression

p(y_{1},y_{2})={\frac {1}{2\pi {\sqrt {1-\rho ^{2}}}}}e^{-{\frac {y_{1}^{2}+y_{2}^{2}-2\rho y_{1}y_{2}}{2(1-\rho ^{2})}}}

where $y_{1}$ and $y_{2}$ are standard Gaussian random variables with correlation $\phi _{y_{1}y_{2}}=\rho$ .

Assume that $r_{2}=Q(y_{2})$ , the correlation between $y_{1}$ and $r_{2}$ is,

\phi _{y_{1}r_{2}}={\frac {1}{2\pi {\sqrt {1-\rho ^{2}}}}}\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }y_{1}Q(y_{2})e^{-{\frac {y_{1}^{2}+y_{2}^{2}-2\rho y_{1}y_{2}}{2(1-\rho ^{2})}}}\,dy_{1}dy_{2}

.

Since

\int _{-\infty }^{\infty }y_{1}e^{-{\frac {1}{2(1-\rho ^{2})}}y_{1}^{2}+{\frac {\rho y_{2}}{1-\rho ^{2}}}y_{1}}\,dy_{1}=\rho {\sqrt {2\pi (1-\rho ^{2})}}y_{2}e^{\frac {\rho ^{2}y_{2}^{2}}{2(1-\rho ^{2})}}

,

the correlation $\phi _{y_{1}r_{2}}$ may be simplified as

\phi _{y_{1}r_{2}}={\frac {\rho }{\sqrt {2\pi }}}\int _{-\infty }^{\infty }y_{2}Q(y_{2})e^{-{\frac {y_{2}^{2}}{2}}}\,dy_{2}

.

The integral above is seen to depend only on the distortion characteristic $Q()$ and is independent of $\rho$ .

Remembering that $\rho =\phi _{y_{1}y_{2}}$ , we observe that for a given distortion characteristic $Q()$ , the ratio ${\frac {\phi _{y_{1}r_{2}}}{\phi _{y_{1}y_{2}}}}$ is $K_{Q}={\frac {1}{\sqrt {2\pi }}}\int _{-\infty }^{\infty }y_{2}Q(y_{2})e^{-{\frac {y_{2}^{2}}{2}}}\,dy_{2}$ .

Therefore, the correlation can be rewritten in the form

$\phi _{y_{1}r_{2}}=K_{Q}\phi _{y_{1}y_{2}}$ .

The above equation is the mathematical expression of the stated "Bussgang‘s theorem".

If $Q(x)={\text{sign}}(x)$ , or called one-bit quantization, then $K_{Q}={\frac {2}{\sqrt {2\pi }}}\int _{0}^{\infty }y_{2}e^{-{\frac {y_{2}^{2}}{2}}}\,dy_{2}={\sqrt {\frac {2}{\pi }}}$ .

^[2]^[3]^[1]^[4]

Arcsine law

If the two random variables are both distorted, i.e., $r_{1}=Q(y_{1}),r_{2}=Q(y_{2})$ , the correlation of $r_{1}$ and $r_{2}$ is

$\phi _{r_{1}r_{2}}=\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }Q(y_{1})Q(y_{2})p(y_{1},y_{2})\,dy_{1}dy_{2}$ .

When $Q(x)={\text{sign}}(x)$ , the expression becomes,

$\phi _{r_{1}r_{2}}={\frac {1}{2\pi {\sqrt {1-\rho ^{2}}}}}\left[\int _{0}^{\infty }\int _{0}^{\infty }e^{-\alpha }\,dy_{1}dy_{2}+\int _{-\infty }^{0}\int _{-\infty }^{0}e^{-\alpha }\,dy_{1}dy_{2}-\int _{0}^{\infty }\int _{-\infty }^{0}e^{-\alpha }\,dy_{1}dy_{2}-\int _{-\infty }^{0}\int _{0}^{\infty }e^{-\alpha }\,dy_{1}dy_{2}\right]$

where $\alpha ={\frac {y_{1}^{2}+y_{2}^{2}-2\rho y_{1}y_{2}}{2(1-\rho ^{2})}}$ .

Noticing that

$\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }p(y_{1},y_{2})\,dy_{1}dy_{2}={\frac {1}{2\pi {\sqrt {1-\rho ^{2}}}}}\left[\int _{0}^{\infty }\int _{0}^{\infty }e^{-\alpha }\,dy_{1}dy_{2}+\int _{-\infty }^{0}\int _{-\infty }^{0}e^{-\alpha }\,dy_{1}dy_{2}+\int _{0}^{\infty }\int _{-\infty }^{0}e^{-\alpha }\,dy_{1}dy_{2}+\int _{-\infty }^{0}\int _{0}^{\infty }e^{-\alpha }\,dy_{1}dy_{2}\right]=1$ ,

and $\int _{0}^{\infty }\int _{0}^{\infty }e^{-\alpha }\,dy_{1}dy_{2}=\int _{-\infty }^{0}\int _{-\infty }^{0}e^{-\alpha }\,dy_{1}dy_{2}$ , $\int _{0}^{\infty }\int _{-\infty }^{0}e^{-\alpha }\,dy_{1}dy_{2}=\int _{-\infty }^{0}\int _{0}^{\infty }e^{-\alpha }\,dy_{1}dy_{2}$ ,

we can simplify the expression of $\phi _{r_{1}r_{2}}$ as

$\phi _{r_{1}r_{2}}={\frac {4}{2\pi {\sqrt {1-\rho ^{2}}}}}\int _{0}^{\infty }\int _{0}^{\infty }e^{-\alpha }\,dy_{1}dy_{2}-1$

Also, it is convenient to introduce the polar coordinate $y_{1}=R\cos \theta ,y_{2}=R\sin \theta$ . It is thus found that

$\phi _{r_{1}r_{2}}={\frac {4}{2\pi {\sqrt {1-\rho ^{2}}}}}\int _{0}^{\pi /2}\int _{0}^{\infty }e^{-{\frac {R^{2}-2R^{2}\rho \cos \theta \sin \theta \ }{2(1-\rho ^{2})}}}R\,dRd\theta -1={\frac {4}{2\pi {\sqrt {1-\rho ^{2}}}}}\int _{0}^{\pi /2}\int _{0}^{\infty }e^{-{\frac {R^{2}(1-\rho \sin 2\theta )}{2(1-\rho ^{2})}}}R\,dRd\theta -1$ .

Integration gives

$\phi _{r_{1}r_{2}}={\frac {2{\sqrt {1-\rho ^{2}}}}{\pi }}\int _{0}^{\pi /2}{\frac {d\theta }{1-\rho \sin 2\theta }}-1=-{\frac {2}{\pi }}\arctan \left({\frac {\rho -\tan \theta }{\sqrt {1-\rho ^{2}}}}\right){\Bigg |}_{0}^{\pi /2}-1={\frac {2}{\pi }}\arcsin(\rho )$ ，

This is called "Arcsine law", which was first found by J. H. Van Vleck in 1943 and republished in 1966.^[2]^[3] The "Arcsine law" can also be proved in a simpler way by applying Price's Theorem.^[4]^[5]

The function $f(x)={\frac {2}{\pi }}\arcsin x$ can be approximated as $f(x)\approx {\frac {2}{\pi }}x$ when $x$ is small.

Price's Theorem

Given two jointly normal random variables $y_{1}$ and $y_{2}$ with joint probability function

${\displaystyle p(y_{1},y_{2})={\frac {1}{2\pi {\sqrt {1-\rho ^{2}}}}}e^{-{\frac {y_{1}^{2}+y_{2}^{2}-2\rho y_{1}y_{2}}{2(1-\rho ^{2})}}}}$ ,

we form the mean

$I(\rho )=E(g(y_{1},y_{2}))=\int _{-\infty }^{+\infty }\int _{-\infty }^{+\infty }g(y_{1},y_{2})p(y_{1},y_{2})\,dy_{1}dy_{2}$

of some function $g(y_{1},y_{2})$ of $(y_{1},y_{2})$ . If $g(y_{1},y_{2})p(y_{1},y_{2})\rightarrow 0$ as $(y_{1},y_{2})\rightarrow 0$ , then

${\frac {\partial ^{n}I(\rho )}{\partial \rho ^{n}}}=\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }{\frac {\partial ^{2n}g(y_{1},y_{2})}{\partial y_{1}^{n}\partial y_{2}^{n}}}p(y_{1},y_{2})\,dy_{1}dy_{2}=E\left({\frac {\partial ^{2n}g(y_{1},y_{2})}{\partial y_{1}^{n}\partial y_{2}^{n}}}\right)$ .

Proof. The joint characteristic function of the random variables $y_{1}$ and $y_{2}$ is by definition the integral

$\Phi (\omega _{1},\omega _{2})=\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }p(y_{1},y_{2})e^{j(\omega _{1}y_{1}+\omega _{2}y_{2})}\,dy_{1}dy_{2}=\exp \left\{-{\frac {\omega _{1}^{2}+\omega _{2}^{2}+2\rho \omega _{1}\omega _{2}}{2}}\right\}$ .

From the two-dimensional inversion formula of Fourier transform, it follows that

$p(y_{1},y_{2})={\frac {1}{4\pi ^{2}}}\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }\Phi (\omega _{1},\omega _{2})e^{-j(\omega _{1}y_{1}+\omega _{2}y_{2})}\,d\omega _{1}d\omega _{2}={\frac {1}{4\pi ^{2}}}\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }\exp \left\{-{\frac {\omega _{1}^{2}+\omega _{2}^{2}+2\rho \omega _{1}\omega _{2}}{2}}\right\}e^{-j(\omega _{1}y_{1}+\omega _{2}y_{2})}\,d\omega _{1}d\omega _{2}$ .

Therefore, plugging the expression of $p(y_{1},y_{2})$ into $I(\rho )$ , and differentiating with respect to $\rho$ , we obtain

${\begin{aligned}{\frac {\partial ^{n}I(\rho )}{\partial \rho ^{n}}}&=\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }g(y_{1},y_{2})p(y_{1},y_{2})\,dy_{1}dy_{2}\\&=\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }g(y_{1},y_{2})\left({\frac {1}{4\pi ^{2}}}\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }{\frac {\partial ^{n}\Phi (\omega _{1},\omega _{2})}{\partial \rho ^{n}}}e^{-j(\omega _{1}y_{1}+\omega _{2}y_{2})}\,d\omega _{1}d\omega _{2}\right)\,dy_{1}dy_{2}\\&=\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }g(y_{1},y_{2})\left({\frac {(-1)^{n}}{4\pi ^{2}}}\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }\omega _{1}^{n}\omega _{2}^{n}\Phi (\omega _{1},\omega _{2})e^{-j(\omega _{1}y_{1}+\omega _{2}y_{2})}\,d\omega _{1}d\omega _{2}\right)\,dy_{1}dy_{2}\\&=\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }g(y_{1},y_{2})\left({\frac {1}{4\pi ^{2}}}\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }\Phi (\omega _{1},\omega _{2}){\frac {\partial ^{2n}e^{-j(\omega _{1}y_{1}+\omega _{2}y_{2})}}{\partial y_{1}^{n}\partial y_{2}^{n}}}\,d\omega _{1}d\omega _{2}\right)\,dy_{1}dy_{2}\\&=\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }g(y_{1},y_{2}){\frac {\partial ^{2n}p(y_{1},y_{2})}{\partial y_{1}^{n}\partial y_{2}^{n}}}\,dy_{1}dy_{2}\\\end{aligned}}$

After repeated integration by parts and using the condition at $\infty$ , we obtain the Price's theorem.

${\begin{aligned}{\frac {\partial ^{n}I(\rho )}{\partial \rho ^{n}}}&=\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }g(y_{1},y_{2}){\frac {\partial ^{2n}p(y_{1},y_{2})}{\partial y_{1}^{n}\partial y_{2}^{n}}}\,dy_{1}dy_{2}\\&=\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }{\frac {\partial ^{2}g(y_{1},y_{2})}{\partial y_{1}\partial y_{2}}}{\frac {\partial ^{2n-2}p(y_{1},y_{2})}{\partial y_{1}^{n-1}\partial y_{2}^{n-1}}}\,dy_{1}dy_{2}\\&=\cdots \\&=\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }{\frac {\partial ^{2n}g(y_{1},y_{2})}{\partial y_{1}^{n}\partial y_{2}^{n}}}p(y_{1},y_{2})\,dy_{1}dy_{2}\end{aligned}}$

^[4]^[5]

Proof of Arcsine law by Price's Theorem

If $g(y_{1},y_{2})={\text{sign}}(y_{1}){\text{sign}}(y_{2})$ , then ${\frac {\partial ^{2}g(y_{1},y_{2})}{\partial y_{1}\partial y_{2}}}=4\delta (y_{1})\delta (y_{2})$ where $\delta ()$ is the Dirac delta function.

Substituting into Price's Theorem, we obtain,

${\frac {\partial E({\text{sign}}(y_{1}){\text{sign}}(y_{2}))}{\partial \rho }}={\frac {\partial I(\rho )}{\partial \rho }}=\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }4\delta (y_{1})\delta (y_{2})p(y_{1},y_{2})\,dy_{1}dy_{2}={\frac {2}{\pi {\sqrt {1-\rho ^{2}}}}}$ .

When $\rho =0$ , $I(\rho )=0$ . Thus

$E\left({\text{sign}}(y_{1}){\text{sign}}(y_{2})\right)=I(\rho )={\frac {2}{\pi }}\int _{0}^{\rho }{\frac {1}{\sqrt {1-\rho ^{2}}}}\,d\rho ={\frac {2}{\pi }}\arcsin(\rho )$ ,

which is Van Vleck's well-known result of "Arcsine law".

^[2]^[3]

Application

This theorem implies that a simplified correlator can be designed. Instead of having to multiply two signals, the cross-correlation problem reduces to the gating of one signal with another.

References

1 2 J.J. Bussgang,"Cross-correlation function of amplitude-distorted Gaussian signals", Res. Lab. Elec., Mas. Inst. Technol., Cambridge MA, Tech. Rep. 216, March 1952.
1 2 3 Vleck, J. H. Van. "The Spectrum of Clipped Noise". Radio Research Laboratory Report of Harvard University. No. 51.
1 2 3 Vleck, J. H. Van; Middleton, D. (January 1966). "The spectrum of clipped noise". Proceedings of the IEEE. 54 (1): 2–19. doi:10.1109/PROC.1966.4567. ISSN 1558-2256.
1 2 3 Price, R. (June 1958). "A useful theorem for nonlinear devices having Gaussian inputs". IRE Transactions on Information Theory. 4 (2): 69–72. doi:10.1109/TIT.1958.1057444. ISSN 2168-2712.
1 2 Papoulis, Athanasios (2002). Probability, Random Variables, and Stochastic Processes. McGraw-Hill. p. 396. ISBN 0-07-366011-6.