Functional correlation

Short description: Dimensionality reduction technique

In statistics, functional correlation is a dimensionality reduction technique used to quantify the correlation and dependence between two variables when the data is functional. Several approaches have been developed to quantify the relation between two functional variables.

Overview

A pair of real valued random functions [math]\displaystyle{ \textstyle X(t) }[/math] and [math]\displaystyle{ \textstyle Y(t) }[/math] with [math]\displaystyle{ t \in T }[/math], a compact interval, can be viewed as realizations of square-integrable stochastic process in a Hilbert space. Since both [math]\displaystyle{ X }[/math] and [math]\displaystyle{ Y }[/math]are infinite dimensional, some kind of dimension reduction is required to explore their relationship. Notions of correlation for functional data include the following.

Functional canonical correlation coefficient (FCCA)

FCCA is a direct extension of multivariate canonical correlation. For a pair of random functions [math]\displaystyle{ X \in \mathcal{L}^2(\mathcal{I_X}) }[/math] and [math]\displaystyle{ Y \in \mathcal{L}^2(\mathcal{I_Y}) }[/math] the first canonical coefficient [math]\displaystyle{ \rho_1 }[/math] is defined as:

[math]\displaystyle{ \begin{align}\rho_1 &= \underset {u \in \mathcal{L}^2(\mathcal{I_X}) , v \in \mathcal{L}^2(\mathcal{I_Y})} {\sup} \operatorname{corr} (\langle u, X\rangle , \langle v, Y\rangle ) \\ & = \underset {u \in \mathcal{L}^2(\mathcal{I_X}) , v \in \mathcal{L}^2(\mathcal{I_Y})} {\sup} \frac { \operatorname{cov} (\langle u, X\rangle , \langle v, Y\rangle )}{\sqrt{\operatorname{var}({\langle u, X\rangle})} \sqrt{\operatorname{var}({\langle v, Y\rangle})}} \\ & = \operatorname{corr} (\langle u_1, X\rangle , \langle v_1, Y\rangle ) \end{align} }[/math]

(1)

where [math]\displaystyle{ \langle \cdot,\cdot\rangle }[/math] denotes the inner product in Lp space (p=2) i.e. [math]\displaystyle{ \langle f_1,f_2\rangle = \int_{\mathcal{I}} f_1(t) f_2(t) \, dt, }[/math] [math]\displaystyle{ \quad f_1, f_2 \in \mathcal{L}^2(\mathcal{I}) }[/math]

The [math]\displaystyle{ k^{th} }[/math] canonical coefficient [math]\displaystyle{ \rho_k }[/math], given [math]\displaystyle{ \rho_1, \rho_2, \ldots, \rho_{k-1} }[/math] is defined as:

[math]\displaystyle{ \rho_k = \underset{\begin{matrix}u \in \mathcal{L}^2(\mathcal{I_X}), v \in \mathcal{L}^2(\mathcal{I_Y})\\ u \perp u_1,\ldots,u_{k-1}, v \perp v_1,\ldots,v_{k-1} \end{matrix}} {\sup} \frac {\operatorname{cov} (\langle u, X\rangle , \langle v, Y\rangle )}{\sqrt{\operatorname{var}(\langle u, X\rangle)} \sqrt{\operatorname{var}(\langle v, Y\rangle)} } = \operatorname{corr} (\langle u_k, X\rangle , \langle v_k, Y\rangle ) }[/math]

(2)

where [math]\displaystyle{ (U_k, V_k)=(\langle u_k, X\rangle , \langle v_k, Y\rangle ) }[/math] is uncorrelated with all previous pairs [math]\displaystyle{ (U_j, V_j)=(\langle u_j, X\rangle , \langle v_j, Y\rangle )_{j=1,2,\ldots,k-1} }[/math].

Thus FCCA implements projections in the directions of [math]\displaystyle{ U_k }[/math] and [math]\displaystyle{ V_k }[/math] for [math]\displaystyle{ X }[/math] and [math]\displaystyle{ Y }[/math] respectively, such that their linear combinations (inner products) [math]\displaystyle{ (U_k, V_k) }[/math] are maximally correlated. [math]\displaystyle{ X }[/math] and [math]\displaystyle{ Y }[/math] are uncorrelated if all their canonical correlations are zero, equivalently, if and only if [math]\displaystyle{ \rho_1 =0 }[/math].

Alternative formulation

The cross-covariance operator for two random functions [math]\displaystyle{ X }[/math] and [math]\displaystyle{ Y }[/math] defined as [math]\displaystyle{ \Sigma_{XY} : \mathcal{L}^2(\mathcal{I_X}) \rightarrow \mathcal{L}^2(\mathcal{I_Y}) : \Sigma_{XY} v(t) = \int \operatorname{cov}( X(t), Y(t)) v(s) \, ds ;\quad v \in \mathcal{L}^2(\mathcal{I_Y}) }[/math] and analogously the auto covariance operators for [math]\displaystyle{ X,\Sigma_{XX} }[/math], for [math]\displaystyle{ Y,\Sigma_{YY} }[/math] and using [math]\displaystyle{ \operatorname{cov} (\langle u, X\rangle , \langle v, Y\rangle ) = \langle u, \Sigma_{XY}Y \rangle }[/math], the [math]\displaystyle{ k^{th} }[/math] canonical coefficient [math]\displaystyle{ \rho_k }[/math] in (2) can be re-written as,

[math]\displaystyle{ \rho_k = \underset{\begin{matrix}u \in \mathcal{L}^2(\mathcal{I_X}), v \in \mathcal{L}^2(\mathcal{I_Y})\\ u \perp u_1,\ldots,u_{k-1}, v \perp v_1,\ldots,v_{k-1} \end{matrix}}{\sup} \frac{\langle u, \Sigma_{XY}v \rangle }{\sqrt{\langle u, \Sigma_{XX}u \rangle}\sqrt{\langle v, \Sigma_{YY}v \rangle}}= \frac{\langle u_k, \Sigma_{XY}v_k \rangle }{\sqrt{\langle u_k, \Sigma_{XX}u_k \rangle}\sqrt{\langle v_k, \Sigma_{YY}v_k \rangle}} }[/math]

(3)

,

where [math]\displaystyle{ (U_k, V_k)=(\langle u_k, X\rangle , \langle v_k, Y\rangle ) }[/math] is uncorrelated with all previous pairs [math]\displaystyle{ (U_j, V_j)=(\langle u_j, X\rangle , \langle v_j, Y\rangle )_{j=1,2,..,k-1} }[/math]

Maximizing (3) is equivalent to finding eigenvalues and eigenvectors of the operator [math]\displaystyle{ R= \Sigma^{-1/2}_{XX} \Sigma_{XY} \Sigma^{-1/2}_{YY} }[/math].

Challenges

Since [math]\displaystyle{ \Sigma_{XX} }[/math] and [math]\displaystyle{ \Sigma_{YY} }[/math] are compact operators, the square root of the auto-covariance operator of [math]\displaystyle{ \mathcal{L^2} }[/math] processes may not be invertible. So the existence of [math]\displaystyle{ R }[/math] and hence computing its eigenvalues and eigenvectors is an ill-posed problem. As a consequence of this inverse problem, overfitting may occur which may lead to an unstable correlation coefficient. Due to this inverse problem, [math]\displaystyle{ \rho_1 }[/math] tends to be biased upwards and therefore close to 1 and hence is difficult to interpret. FCCA also requires densely recorded functional data so that the inner products in (2) can be accurately evaluated.

Possible solutions

Some possible solutions to this problem have been discussed.

• By restricting the maximization of (1) to discrete [math]\displaystyle{ l^2 }[/math] sequence spaces that are restricted to a reproducing kernel Hilbert space instead of entire [math]\displaystyle{ \mathcal{L}^2 }[/math]
• Using cross-validation to regularize the FCCA in practical implementation.

Functional singular correlation analysis (FSCA)

FSCA bypasses the inverse problem by simply replacing the objective function by covariance in place of correlation in (2). FSCA aims to quantify the dependency of [math]\displaystyle{ X,Y }[/math] by implementing the concept of functional singular-value decomposition for the cross-covariance operator. FSCA can be viewed as an extension of analyses using singular-value decomposition of vector data to functional data. For a pair of random functions [math]\displaystyle{ X \in \mathcal{L}^2(\mathcal{I_X}) }[/math] and [math]\displaystyle{ Y \in \mathcal{L}^2(\mathcal{I_Y}) }[/math] with smooth mean functions [math]\displaystyle{ \mu_X(t)=\mathbb{E}(X(t)) }[/math] and [math]\displaystyle{ \mu_Y(t)=\mathbb{E}(Y(t)) }[/math] and smooth covariance functions, FSCA aims at a "functional covariance" corresponding to the first singular value of the cross-covariance operator [math]\displaystyle{ \Sigma_{XY} }[/math],

[math]\displaystyle{ \underset {u \in \mathcal{L}^2(\mathcal{I_X}) , v \in \mathcal{L}^2(\mathcal{I_Y})} {\sup} \operatorname{cov} (\langle u, X\rangle , \langle v, Y\rangle )=\underset {u \in \mathcal{L}^2(\mathcal{I_X}) , v \in \mathcal{L}^2(\mathcal{I_Y})} {\sup} \langle u, \Sigma_{XY}v \rangle }[/math]

(4)

which is attained at functions [math]\displaystyle{ u_1 \in \mathcal{L}^2(\mathcal{I_X}) , v_1 \in \mathcal{L}^2(\mathcal{I_Y}) }[/math]. A standardized version of this serves as a functional correlation and is defined as

[math]\displaystyle{ \rho_1 = \frac{\langle u_1, \Sigma_{XY}v_1 \rangle}{\sqrt{\langle u_1, \Sigma_{XX}u_1 \rangle} \sqrt{\langle v_1, \Sigma_{YY}v_1 \rangle}} }[/math]

(5)

,

The singular representation of the cross-covariance can be employed to find a solution to the maximization problem (4). Analogously, we can extend this concept to find the next [math]\displaystyle{ k }[/math] ordered singular correlation coefficients [math]\displaystyle{ \rho_1, \rho_2,\ldots,\rho_k }[/math].

Correlation as angle between functions

In the multivariate case, the inner product of two vectors [math]\displaystyle{ a }[/math] and [math]\displaystyle{ b }[/math] is defined as, [math]\displaystyle{ \langle a,b \rangle = \|a\| \|b\| \cos\alpha }[/math] where [math]\displaystyle{ \alpha }[/math] is the angle between [math]\displaystyle{ a }[/math] and [math]\displaystyle{ b }[/math]. This can be extended to the space of square integrable random functions. For this notion to be a meaningful measure of alignment of shapes, the integrals of the functions, which are the projections on the constant function 1, are subtracted. This part corresponds to a "static part" and the remainder can be thought of as a "dynamic part" for each random function. The cosine of the [math]\displaystyle{ L^2 }[/math] angle between these "dynamic parts" then provides a correlation measure of functional shapes. Denoting [math]\displaystyle{ M_1=\langle X,1 \rangle }[/math] and [math]\displaystyle{ M_2=\langle Y,1 \rangle }[/math] the standardized curves may be defined as [math]\displaystyle{ X^*(t)=\left(X(t)-M_1(t)\right) \Big / \left(\int (X(t)-M_1(t))^2 \, dt\right)^{1/2} }[/math] [math]\displaystyle{ Y^*(t)=\left(Y(t)-M_2(t)\right) \Big / \left(\int (Y(t)-M_2(t))^2 \,dt\right)^{1/2} }[/math] and the correlation is defined as,

[math]\displaystyle{ \rho = \mathbb{E} \langle X^*, Y^* \rangle }[/math]

(6)

See also

• Functional data analysis
• Functional principal component analysis
• Karhunen–Loève theorem
• Functional regression
• Generalized functional linear model
• Stochastic processes
• Canonical correlation