# Cross-covariance matrix

Short description: Type of matrix in probability theory and statistics

In probability theory and statistics, a cross-covariance matrix is a matrix whose element in the i, j position is the covariance between the i-th element of a random vector and j-th element of another random vector. A random vector is a random variable with multiple dimensions. Each element of the vector is a scalar random variable. Each element has either a finite number of observed empirical values or a finite or infinite number of potential values. The potential values are specified by a theoretical joint probability distribution. Intuitively, the cross-covariance matrix generalizes the notion of covariance to multiple dimensions.

The cross-covariance matrix of two random vectors $\displaystyle{ \mathbf{X} }$ and $\displaystyle{ \mathbf{Y} }$ is typically denoted by $\displaystyle{ \operatorname{K}_{\mathbf{X}\mathbf{Y}} }$ or $\displaystyle{ \Sigma_{\mathbf{X}\mathbf{Y}} }$.

## Definition

For random vectors $\displaystyle{ \mathbf{X} }$ and $\displaystyle{ \mathbf{Y} }$, each containing random elements whose expected value and variance exist, the cross-covariance matrix of $\displaystyle{ \mathbf{X} }$ and $\displaystyle{ \mathbf{Y} }$ is defined by[1]:p.336

$\displaystyle{ \operatorname{K}_{\mathbf{X}\mathbf{Y}} = \operatorname{cov}(\mathbf{X},\mathbf{Y}) \stackrel{\mathrm{def}}{=}\ \operatorname{E}[(\mathbf{X}-\mathbf{\mu_X})(\mathbf{Y}-\mathbf{\mu_Y})^{\rm T}] }$

(Eq.1)

where $\displaystyle{ \mathbf{\mu_X} = \operatorname{E}[\mathbf{X}] }$ and $\displaystyle{ \mathbf{\mu_Y} = \operatorname{E}[\mathbf{Y}] }$ are vectors containing the expected values of $\displaystyle{ \mathbf{X} }$ and $\displaystyle{ \mathbf{Y} }$. The vectors $\displaystyle{ \mathbf{X} }$ and $\displaystyle{ \mathbf{Y} }$ need not have the same dimension, and either might be a scalar value.

The cross-covariance matrix is the matrix whose $\displaystyle{ (i,j) }$ entry is the covariance

$\displaystyle{ \operatorname{K}_{X_i Y_j} = \operatorname{cov}[X_i, Y_j] = \operatorname{E}[(X_i - \operatorname{E}[X_i])(Y_j - \operatorname{E}[Y_j])] }$

between the i-th element of $\displaystyle{ \mathbf{X} }$ and the j-th element of $\displaystyle{ \mathbf{Y} }$. This gives the following component-wise definition of the cross-covariance matrix.

$\displaystyle{ \operatorname{K}_{\mathbf{X}\mathbf{Y}}= \begin{bmatrix} \mathrm{E}[(X_1 - \operatorname{E}[X_1])(Y_1 - \operatorname{E}[Y_1])] & \mathrm{E}[(X_1 - \operatorname{E}[X_1])(Y_2 - \operatorname{E}[Y_2])] & \cdots & \mathrm{E}[(X_1 - \operatorname{E}[X_1])(Y_n - \operatorname{E}[Y_n])] \\ \\ \mathrm{E}[(X_2 - \operatorname{E}[X_2])(Y_1 - \operatorname{E}[Y_1])] & \mathrm{E}[(X_2 - \operatorname{E}[X_2])(Y_2 - \operatorname{E}[Y_2])] & \cdots & \mathrm{E}[(X_2 - \operatorname{E}[X_2])(Y_n - \operatorname{E}[Y_n])] \\ \\ \vdots & \vdots & \ddots & \vdots \\ \\ \mathrm{E}[(X_m - \operatorname{E}[X_m])(Y_1 - \operatorname{E}[Y_1])] & \mathrm{E}[(X_m - \operatorname{E}[X_m])(Y_2 - \operatorname{E}[Y_2])] & \cdots & \mathrm{E}[(X_m - \operatorname{E}[X_m])(Y_n - \operatorname{E}[Y_n])] \end{bmatrix} }$

## Example

For example, if $\displaystyle{ \mathbf{X} = \left( X_1,X_2,X_3 \right)^{\rm T} }$ and $\displaystyle{ \mathbf{Y} = \left( Y_1,Y_2 \right)^{\rm T} }$ are random vectors, then $\displaystyle{ \operatorname{cov}(\mathbf{X},\mathbf{Y}) }$ is a $\displaystyle{ 3 \times 2 }$ matrix whose $\displaystyle{ (i,j) }$-th entry is $\displaystyle{ \operatorname{cov}(X_i,Y_j) }$.

## Properties

For the cross-covariance matrix, the following basic properties apply:[2]

1. $\displaystyle{ \operatorname{cov}(\mathbf{X},\mathbf{Y}) = \operatorname{E}[\mathbf{X} \mathbf{Y}^{\rm T}] - \mathbf{\mu_X} \mathbf{\mu_Y}^{\rm T} }$
2. $\displaystyle{ \operatorname{cov}(\mathbf{X},\mathbf{Y}) = \operatorname{cov}(\mathbf{Y},\mathbf{X})^{\rm T} }$
3. $\displaystyle{ \operatorname{cov}(\mathbf{X_1} + \mathbf{X_2},\mathbf{Y}) = \operatorname{cov}(\mathbf{X_1},\mathbf{Y}) + \operatorname{cov}(\mathbf{X_2}, \mathbf{Y}) }$
4. $\displaystyle{ \operatorname{cov}(A\mathbf{X}+ \mathbf{a}, B^{\rm T}\mathbf{Y} + \mathbf{b}) = A\, \operatorname{cov}(\mathbf{X}, \mathbf{Y}) \,B }$
5. If $\displaystyle{ \mathbf{X} }$ and $\displaystyle{ \mathbf{Y} }$ are independent (or somewhat less restrictedly, if every random variable in $\displaystyle{ \mathbf{X} }$ is uncorrelated with every random variable in $\displaystyle{ \mathbf{Y} }$), then $\displaystyle{ \operatorname{cov}(\mathbf{X},\mathbf{Y}) = 0_{p\times q} }$

where $\displaystyle{ \mathbf{X} }$, $\displaystyle{ \mathbf{X_1} }$ and $\displaystyle{ \mathbf{X_2} }$ are random $\displaystyle{ p \times 1 }$ vectors, $\displaystyle{ \mathbf{Y} }$ is a random $\displaystyle{ q \times 1 }$ vector, $\displaystyle{ \mathbf{a} }$ is a $\displaystyle{ q \times 1 }$ vector, $\displaystyle{ \mathbf{b} }$ is a $\displaystyle{ p \times 1 }$ vector, $\displaystyle{ A }$ and $\displaystyle{ B }$ are $\displaystyle{ q \times p }$ matrices of constants, and $\displaystyle{ 0_{p\times q} }$ is a $\displaystyle{ p \times q }$ matrix of zeroes.

## Definition for complex random vectors

If $\displaystyle{ \mathbf{Z} }$ and $\displaystyle{ \mathbf{W} }$ are complex random vectors, the definition of the cross-covariance matrix is slightly changed. Transposition is replaced by Hermitian transposition:

$\displaystyle{ \operatorname{K}_{\mathbf{Z}\mathbf{W}} = \operatorname{cov}(\mathbf{Z},\mathbf{W}) \stackrel{\mathrm{def}}{=}\ \operatorname{E}[(\mathbf{Z}-\mathbf{\mu_Z})(\mathbf{W}-\mathbf{\mu_W})^{\rm H}] }$

For complex random vectors, another matrix called the pseudo-cross-covariance matrix is defined as follows:

$\displaystyle{ \operatorname{J}_{\mathbf{Z}\mathbf{W}} = \operatorname{cov}(\mathbf{Z},\overline{\mathbf{W}}) \stackrel{\mathrm{def}}{=}\ \operatorname{E}[(\mathbf{Z}-\mathbf{\mu_Z})(\mathbf{W}-\mathbf{\mu_W})^{\rm T}] }$

## Uncorrelatedness

Main page: Uncorrelatedness (probability theory)

Two random vectors $\displaystyle{ \mathbf{X} }$ and $\displaystyle{ \mathbf{Y} }$ are called uncorrelated if their cross-covariance matrix $\displaystyle{ \operatorname{K}_{\mathbf{X}\mathbf{Y}} }$ matrix is a zero matrix.[1]:p.337

Complex random vectors $\displaystyle{ \mathbf{Z} }$ and $\displaystyle{ \mathbf{W} }$ are called uncorrelated if their covariance matrix and pseudo-covariance matrix is zero, i.e. if $\displaystyle{ \operatorname{K}_{\mathbf{Z}\mathbf{W}} = \operatorname{J}_{\mathbf{Z}\mathbf{W}} = 0 }$.

## References

1. Gubner, John A. (2006). Probability and Random Processes for Electrical and Computer Engineers. Cambridge University Press. ISBN 978-0-521-86470-1.
2. Taboga, Marco (2010). "Lectures on probability theory and mathematical statistics".