Tucker decomposition

Short description: Tensor decomposition

In mathematics, Tucker decomposition decomposes a tensor into a set of matrices and one small core tensor. It is named after Ledyard R. Tucker although it goes back to Hitchcock in 1927. Initially described as a three-mode extension of factor analysis and principal component analysis it may actually be generalized to higher mode analysis, which is also called higher-order singular value decomposition (HOSVD).

It may be regarded as a more flexible PARAFAC (parallel factor analysis) model. In PARAFAC the core tensor is restricted to be "diagonal".

In practice, Tucker decomposition is used as a modelling tool. For instance, it is used to model three-way (or higher way) data by means of relatively small numbers of components for each of the three or more modes, and the components are linked to each other by a three- (or higher-) way core array. The model parameters are estimated in such a way that, given fixed numbers of components, the modelled data optimally resemble the actual data in the least squares sense. The model gives a summary of the information in the data, in the same way as principal components analysis does for two-way data.

For a 3rd-order tensor [math]\displaystyle{ T \in F^{n_{1} \times n_{2} \times n_{3}} }[/math], where [math]\displaystyle{ F }[/math] is either [math]\displaystyle{ \mathbb{R} }[/math] or [math]\displaystyle{ \mathbb{C} }[/math], Tucker Decomposition can be denoted as follows, [math]\displaystyle{ T = \mathcal{T} \times_{1} U^{(1)} \times_{2} U^{(2)} \times_{3} U^{(3)} }[/math] where [math]\displaystyle{ \mathcal{T} \in F^{d_{1} \times d_{2} \times d_{3}} }[/math] is the core tensor, a 3rd-order tensor that contains the 1-mode, 2-mode and 3-mode singular values of [math]\displaystyle{ T }[/math], which are defined as the Frobenius norm of the 1-mode, 2-mode and 3-mode slices of tensor [math]\displaystyle{ \mathcal{T} }[/math] respectively. [math]\displaystyle{ U^{(1)}, U^{(2)}, U^{(3)} }[/math] are unitary matrices in [math]\displaystyle{ F^{d_{1} \times n_{1}}, F^{d_{2} \times n_{2}}, F^{d_{3} \times n_{3}} }[/math] respectively. The j-mode product (j = 1, 2, 3) of [math]\displaystyle{ \mathcal{T} }[/math] by [math]\displaystyle{ U^{(j)} }[/math] is denoted as [math]\displaystyle{ \mathcal{T} \times U^{(j)} }[/math] with entries as

[math]\displaystyle{ \begin{align} (\mathcal{T} \times_{1} U^{(1)})(n_{1}, d_{2}, d_{3}) &= \sum_{i_{1}=1}^{d_{1}} \mathcal{T}(i_{1}, d_{2}, d_{3})U^{(1)}(i_{1}, n_{1}) \\ (\mathcal{T} \times_{2} U^{(2)})(d_{1}, n_{2}, d_{3}) &= \sum_{i_{2}=1}^{d_{2}} \mathcal{T}(d_{1}, i_{2}, d_{3})U^{(2)}(i_{2}, n_{2}) \\ (\mathcal{T} \times_{3} U^{(3)})(d_{1}, d_{2}, n_{3}) &= \sum_{i_{3}=1}^{d_{3}} \mathcal{T}(d_{1}, d_{2}, i_{3})U^{(3)}(i_{3}, n_{3}) \end{align} }[/math]

Taking [math]\displaystyle{ d_i = n_i }[/math] for all [math]\displaystyle{ i }[/math] is always sufficient to represent [math]\displaystyle{ T }[/math] exactly, but often [math]\displaystyle{ T }[/math] can be compressed or efficiently approximately by choosing [math]\displaystyle{ d_i \lt n_i }[/math]. A common choice is [math]\displaystyle{ d_1 = d_2 = d_3 = \min(n_1, n_2, n_3) }[/math], which can be effective when the difference in dimension sizes is large.

There are two special cases of Tucker decomposition:

Tucker1: if [math]\displaystyle{ U^{(2)} }[/math] and [math]\displaystyle{ U^{(3)} }[/math] are identity, then [math]\displaystyle{ T = \mathcal{T} \times_{1} U^{(1)} }[/math]

Tucker2: if [math]\displaystyle{ U^{(3)} }[/math] is identity, then [math]\displaystyle{ T = \mathcal{T} \times_{1} U^{(1)} \times_{2} U^{(2)} }[/math] .

RESCAL decomposition can be seen as a special case of Tucker where [math]\displaystyle{ U^{(3)} }[/math] is identity and [math]\displaystyle{ U^{(1)} }[/math] is equal to [math]\displaystyle{ U^{(2)} }[/math] .

See also

• Higher-order singular value decomposition
• Multilinear principal component analysis