Cross-covariance matrix

Not to be confused with Covariance matrix.

Short description: Type of matrix in probability theory and statistics

In probability theory and statistics, a cross-covariance matrix is a matrix whose element in the i, j position is the covariance between the i-th element of a random vector and j-th element of another random vector. A random vector is a random variable with multiple dimensions. Each element of the vector is a scalar random variable. Each element has either a finite number of observed empirical values or a finite or infinite number of potential values. The potential values are specified by a theoretical joint probability distribution. Intuitively, the cross-covariance matrix generalizes the notion of covariance to multiple dimensions.

The cross-covariance matrix of two random vectors [math]\displaystyle{ \mathbf{X} }[/math] and [math]\displaystyle{ \mathbf{Y} }[/math] is typically denoted by [math]\displaystyle{ \operatorname{K}_{\mathbf{X}\mathbf{Y}} }[/math] or [math]\displaystyle{ \Sigma_{\mathbf{X}\mathbf{Y}} }[/math].

Definition

For random vectors [math]\displaystyle{ \mathbf{X} }[/math] and [math]\displaystyle{ \mathbf{Y} }[/math], each containing random elements whose expected value and variance exist, the cross-covariance matrix of [math]\displaystyle{ \mathbf{X} }[/math] and [math]\displaystyle{ \mathbf{Y} }[/math] is defined by

[math]\displaystyle{ \operatorname{K}_{\mathbf{X}\mathbf{Y}} = \operatorname{cov}(\mathbf{X},\mathbf{Y}) \stackrel{\mathrm{def}}{=}\ \operatorname{E}[(\mathbf{X}-\mathbf{\mu_X})(\mathbf{Y}-\mathbf{\mu_Y})^{\rm T}] }[/math]

(Eq.1)

where [math]\displaystyle{ \mathbf{\mu_X} = \operatorname{E}[\mathbf{X}] }[/math] and [math]\displaystyle{ \mathbf{\mu_Y} = \operatorname{E}[\mathbf{Y}] }[/math] are vectors containing the expected values of [math]\displaystyle{ \mathbf{X} }[/math] and [math]\displaystyle{ \mathbf{Y} }[/math]. The vectors [math]\displaystyle{ \mathbf{X} }[/math] and [math]\displaystyle{ \mathbf{Y} }[/math] need not have the same dimension, and either might be a scalar value.

The cross-covariance matrix is the matrix whose [math]\displaystyle{ (i,j) }[/math] entry is the covariance

[math]\displaystyle{ \operatorname{K}_{X_i Y_j} = \operatorname{cov}[X_i, Y_j] = \operatorname{E}[(X_i - \operatorname{E}[X_i])(Y_j - \operatorname{E}[Y_j])] }[/math]

between the i-th element of [math]\displaystyle{ \mathbf{X} }[/math] and the j-th element of [math]\displaystyle{ \mathbf{Y} }[/math]. This gives the following component-wise definition of the cross-covariance matrix.

[math]\displaystyle{ \operatorname{K}_{\mathbf{X}\mathbf{Y}}= \begin{bmatrix} \mathrm{E}[(X_1 - \operatorname{E}[X_1])(Y_1 - \operatorname{E}[Y_1])] & \mathrm{E}[(X_1 - \operatorname{E}[X_1])(Y_2 - \operatorname{E}[Y_2])] & \cdots & \mathrm{E}[(X_1 - \operatorname{E}[X_1])(Y_n - \operatorname{E}[Y_n])] \\ \\ \mathrm{E}[(X_2 - \operatorname{E}[X_2])(Y_1 - \operatorname{E}[Y_1])] & \mathrm{E}[(X_2 - \operatorname{E}[X_2])(Y_2 - \operatorname{E}[Y_2])] & \cdots & \mathrm{E}[(X_2 - \operatorname{E}[X_2])(Y_n - \operatorname{E}[Y_n])] \\ \\ \vdots & \vdots & \ddots & \vdots \\ \\ \mathrm{E}[(X_m - \operatorname{E}[X_m])(Y_1 - \operatorname{E}[Y_1])] & \mathrm{E}[(X_m - \operatorname{E}[X_m])(Y_2 - \operatorname{E}[Y_2])] & \cdots & \mathrm{E}[(X_m - \operatorname{E}[X_m])(Y_n - \operatorname{E}[Y_n])] \end{bmatrix} }[/math]

Example

For example, if [math]\displaystyle{ \mathbf{X} = \left( X_1,X_2,X_3 \right)^{\rm T} }[/math] and [math]\displaystyle{ \mathbf{Y} = \left( Y_1,Y_2 \right)^{\rm T} }[/math] are random vectors, then [math]\displaystyle{ \operatorname{cov}(\mathbf{X},\mathbf{Y}) }[/math] is a [math]\displaystyle{ 3 \times 2 }[/math] matrix whose [math]\displaystyle{ (i,j) }[/math]-th entry is [math]\displaystyle{ \operatorname{cov}(X_i,Y_j) }[/math].

Properties

For the cross-covariance matrix, the following basic properties apply:

1. [math]\displaystyle{ \operatorname{cov}(\mathbf{X},\mathbf{Y}) = \operatorname{E}[\mathbf{X} \mathbf{Y}^{\rm T}] - \mathbf{\mu_X} \mathbf{\mu_Y}^{\rm T} }[/math]
2. [math]\displaystyle{ \operatorname{cov}(\mathbf{X},\mathbf{Y}) = \operatorname{cov}(\mathbf{Y},\mathbf{X})^{\rm T} }[/math]
3. [math]\displaystyle{ \operatorname{cov}(\mathbf{X_1} + \mathbf{X_2},\mathbf{Y}) = \operatorname{cov}(\mathbf{X_1},\mathbf{Y}) + \operatorname{cov}(\mathbf{X_2}, \mathbf{Y}) }[/math]
4. [math]\displaystyle{ \operatorname{cov}(A\mathbf{X}+ \mathbf{a}, B^{\rm T}\mathbf{Y} + \mathbf{b}) = A\, \operatorname{cov}(\mathbf{X}, \mathbf{Y}) \,B }[/math]
5. If [math]\displaystyle{ \mathbf{X} }[/math] and [math]\displaystyle{ \mathbf{Y} }[/math] are independent (or somewhat less restrictedly, if every random variable in [math]\displaystyle{ \mathbf{X} }[/math] is uncorrelated with every random variable in [math]\displaystyle{ \mathbf{Y} }[/math]), then [math]\displaystyle{ \operatorname{cov}(\mathbf{X},\mathbf{Y}) = 0_{p\times q} }[/math]

where [math]\displaystyle{ \mathbf{X} }[/math], [math]\displaystyle{ \mathbf{X_1} }[/math] and [math]\displaystyle{ \mathbf{X_2} }[/math] are random [math]\displaystyle{ p \times 1 }[/math] vectors, [math]\displaystyle{ \mathbf{Y} }[/math] is a random [math]\displaystyle{ q \times 1 }[/math] vector, [math]\displaystyle{ \mathbf{a} }[/math] is a [math]\displaystyle{ q \times 1 }[/math] vector, [math]\displaystyle{ \mathbf{b} }[/math] is a [math]\displaystyle{ p \times 1 }[/math] vector, [math]\displaystyle{ A }[/math] and [math]\displaystyle{ B }[/math] are [math]\displaystyle{ q \times p }[/math] matrices of constants, and [math]\displaystyle{ 0_{p\times q} }[/math] is a [math]\displaystyle{ p \times q }[/math] matrix of zeroes.

Definition for complex random vectors

If [math]\displaystyle{ \mathbf{Z} }[/math] and [math]\displaystyle{ \mathbf{W} }[/math] are complex random vectors, the definition of the cross-covariance matrix is slightly changed. Transposition is replaced by Hermitian transposition:

[math]\displaystyle{ \operatorname{K}_{\mathbf{Z}\mathbf{W}} = \operatorname{cov}(\mathbf{Z},\mathbf{W}) \stackrel{\mathrm{def}}{=}\ \operatorname{E}[(\mathbf{Z}-\mathbf{\mu_Z})(\mathbf{W}-\mathbf{\mu_W})^{\rm H}] }[/math]

For complex random vectors, another matrix called the pseudo-cross-covariance matrix is defined as follows:

[math]\displaystyle{ \operatorname{J}_{\mathbf{Z}\mathbf{W}} = \operatorname{cov}(\mathbf{Z},\overline{\mathbf{W}}) \stackrel{\mathrm{def}}{=}\ \operatorname{E}[(\mathbf{Z}-\mathbf{\mu_Z})(\mathbf{W}-\mathbf{\mu_W})^{\rm T}] }[/math]

Uncorrelatedness

Two random vectors [math]\displaystyle{ \mathbf{X} }[/math] and [math]\displaystyle{ \mathbf{Y} }[/math] are called uncorrelated if their cross-covariance matrix [math]\displaystyle{ \operatorname{K}_{\mathbf{X}\mathbf{Y}} }[/math] matrix is a zero matrix.

Complex random vectors [math]\displaystyle{ \mathbf{Z} }[/math] and [math]\displaystyle{ \mathbf{W} }[/math] are called uncorrelated if their covariance matrix and pseudo-covariance matrix is zero, i.e. if [math]\displaystyle{ \operatorname{K}_{\mathbf{Z}\mathbf{W}} = \operatorname{J}_{\mathbf{Z}\mathbf{W}} = 0 }[/math].