Tensor product model transformation

Short description: Key concept in higher-order singular value decomposition of functions

In mathematics, the tensor product (TP) model transformation was proposed by Baranyi and Yam as key concept for higher-order singular value decomposition of functions. It transforms a function (which can be given via closed formulas or neural networks, fuzzy logic, etc.) into TP function form if such a transformation is possible. If an exact transformation is not possible, then the method determines a TP function that is an approximation of the given function. Hence, the TP model transformation can provide a trade-off between approximation accuracy and complexity.

A free MATLAB implementation of the TP model transformation can be downloaded at [1] or an old version of the toolbox is available at MATLAB Central [2]. A key underpinning of the transformation is the higher-order singular value decomposition.

Besides being a transformation of functions, the TP model transformation is also a new concept in qLPV based control which plays a central role in the providing a valuable means of bridging between identification and polytopic systems theories. The TP model transformation is uniquely effective in manipulating the convex hull of polytopic forms, and, as a result has revealed and proved the fact that convex hull manipulation is a necessary and crucial step in achieving optimal solutions and decreasing conservativeness in modern LMI based control theory. Thus, although it is a transformation in a mathematical sense, it has established a conceptually new direction in control theory and has laid the ground for further new approaches towards optimality. Further details on the control theoretical aspects of the TP model transformation can be found here: TP model transformation in control theory.

The TP model transformation motivated the definition of the "HOSVD canonical form of TP functions", on which further information can be found here. It has been proved that the TP model transformation is capable of numerically reconstructing this HOSVD based canonical form. Thus, the TP model transformation can be viewed as a numerical method to compute the HOSVD of functions, which provides exact results if the given function has a TP function structure and approximative results otherwise.

The TP model transformation has recently been extended in order to derive various types of convex TP functions and to manipulate them. This feature has led to new optimization approaches in qLPV system analysis and design, as described at TP model transformation in control theory.

Definitions

Finite element TP function

A given function [math]\displaystyle{ f({\mathbf{x}}) }[/math], where [math]\displaystyle{ \mathbf{x}\in R^N }[/math], is a TP function if it has the structure:

[math]\displaystyle{ f(\mathbf{x})=\sum_{i_1=1}^{I_1} \sum_{i_2=1}^{I_2} \ldots \sum_{i_N=1}^{I_N} \prod_{n=1}^N w_{n,i_n}(x_n) s_{i_1,i_2,\ldots,i_N}, }[/math]

that is, using compact tensor notation (using the tensor product operation [math]\displaystyle{ \otimes }[/math] of ):

[math]\displaystyle{ f(\mathbf{x})=\mathcal{S}\mathop{\otimes}_{n=1}^N\mathbf{w}_n(x_n), }[/math]

where core tensor [math]\displaystyle{ \mathcal{S}\in \mathcal{R}^{I_1\times I_2\times \ldots \times I_N} }[/math] is constructed from [math]\displaystyle{ s_{i_1 i_2 \ldots i_N} }[/math], and row vector [math]\displaystyle{ \mathbf{w}_n(x_n), (n=1 \ldots N) }[/math] contains continuous univariate weighting functions [math]\displaystyle{ w_{n,i_n}(x_n),(i_n=1 \ldots I_n) }[/math]. The function [math]\displaystyle{ w_{n,i_n}(x_n) }[/math] is the [math]\displaystyle{ i_n }[/math]-th weighting function defined on the [math]\displaystyle{ n }[/math]-th dimension, and [math]\displaystyle{ x_n }[/math] is the [math]\displaystyle{ n }[/math]-the element of vector [math]\displaystyle{ \mathbf{x} }[/math]. Finite element means that [math]\displaystyle{ I_n }[/math] is bounded for all [math]\displaystyle{ n }[/math]. For qLPV modelling and control applications a higher structure of TP functions are referred to as TP model.

Finite element TP model (TP model in short)

This is a higher structure of TP function:

[math]\displaystyle{ \mathcal{F}(\mathbf{x})=\mathcal{S}\boxtimes_{n=1}^N\mathbf{w}_n(x_n). }[/math]

Here [math]\displaystyle{ \mathcal{Y}=\mathcal{F}({\mathbf{x}}) }[/math] is a tensor as [math]\displaystyle{ \mathcal{Y}\in \mathcal{R}^{L_1\times L_2\times \ldots L_O} }[/math], thus the size of the core tensor is [math]\displaystyle{ \mathcal{S}\in \mathcal{R}^{I_1\times I_2\times \ldots \times I_N \times L_1\times L_2\times ... \times L_O} }[/math]. The product operator [math]\displaystyle{ \boxtimes }[/math] has the same role as [math]\displaystyle{ \otimes }[/math], but expresses the fact that the tensor product is applied on the [math]\displaystyle{ L_1\times L_2\times ... \times L_O }[/math] sized tensor elements of the core tensor [math]\displaystyle{ \mathcal{S} }[/math]. Vector [math]\displaystyle{ \mathbf{x} }[/math] is an element of the closed hypercube [math]\displaystyle{ \Omega=[a_1,b_1]\times[a_2,b_2]\times ... \times[a_N,b_N]\subset R^N }[/math].

Finite element convex TP function or model

A TP function or model is convex if the weighting functions hold:

[math]\displaystyle{ \forall n : \sum_{i_n=1}^{I_n} w_{n,i_n}(x_n) = 1 }[/math] and [math]\displaystyle{ w_{n,i_n}(x_n) \in [0,1] . }[/math]

This means that [math]\displaystyle{ f(\mathbf{x}) }[/math] is inside the convex hull defined by the core tensor for all [math]\displaystyle{ \mathbf{x} \in \Omega }[/math].

TP model transformation

Assume a given TP model [math]\displaystyle{ \mathcal{Y} = \mathcal{F}(\mathbf{x}) }[/math], where [math]\displaystyle{ \mathbf{x}\in \Omega \subset R^N }[/math], whose TP structure maybe unknown (e.g. it is given by neural networks). The TP model transformation determines its TP structure as

[math]\displaystyle{ \mathcal{F}(\mathbf{x})=\mathcal{S}\boxtimes_{n=1}^N\mathbf{w}_n(x_n) }[/math],

namely it generates the core tensor [math]\displaystyle{ \mathcal{S} }[/math] and the weighting functions [math]\displaystyle{ \mathbf{w}_n(x_n) }[/math] for all [math]\displaystyle{ n=1 \ldots N }[/math]. Its free MATLAB implementation is downloadable at [3] or at MATLAB Central [4].

If the given [math]\displaystyle{ \mathcal{F}(\mathbf{x}) }[/math] does not have TP structure (i.e. it is not in the class of TP models), then the TP model transformation determines its approximation:

[math]\displaystyle{ \mathcal{F}(\mathbf{x}) \approx \mathcal{S}\boxtimes_{n=1}^N\mathbf{w}_n(x_n), }[/math]

where trade-off is offered by the TP model transformation between complexity (number of components in the core tensor or the number of weighting functions) and the approximation accuracy. The TP model can be generated according to various constrains. Typical TP models generated by the TP model transformation are:

• HOSVD canonical form of TP functions or TP model (qLPV models),
• Various kinds of TP type polytopic form or convex TP model forms (this advantage is used in qLPV system analysis and design).

Properties of the TP model transformation

• It is a non-heuristic and tractable numerical method firstly proposed in control theory.
• It transforms the given function into finite element TP structure. If this structure does not exist, then the transformation gives an approximation under a constraint on the number of elements.
• It can be executed uniformly (irrespective of whether the model is given in the form of analytical equations resulting from physical considerations, or as an outcome of soft computing based identification techniques (such as neural networks or fuzzy logic based methods, or as a result of a black-box identification), without analytical interaction, within a reasonable amount of time. Thus, the transformation replaces the analytical and in many cases complex and not obvious conversions to numerical, tractable, straightforward operations.
• It generates the HOSVD-based canonical form of TP functions, which is a unique representation. It was proven by Szeidl that the TP model transformation numerically reconstructs the HOSVD of functions. This form extracts the unique structure of a given TP function in the same sense as the HOSVD does for tensors and matrices, in a way such that:

• the number of weighting functions are minimized per dimensions (hence the size of the core tensor);
• the weighting functions are one variable functions of the parameter vector in an orthonormed system for each parameter (singular functions);
• the sub tensors of the core tensor are also in orthogonal positions;
• the core tensor and the weighting functions are ordered according to the higher-order singular values of the parameter vector;
• it has a unique form (except for some special cases such as there are equal singular values);
• introduces and defines the rank of the TP function by the dimensions of the parameter vector;

• The above point can be extended to TP models (qLPV models to determine the HOSVD based canonical form of qLPV model to order the main component of the qLPV model). Since the core tensor is [math]\displaystyle{ N+O }[/math] dimensional, but the weighting functions are determined only for dimensions [math]\displaystyle{ n=1 \ldots N }[/math], namely the core tensor is constructed from [math]\displaystyle{ O }[/math] dimensional elements, therefore the resulting TP form is not unique.
• The core step of the TP model transformation was extended to generate different types of convex TP functions or TP models (TP type polytopic qLPV models), in order to focus on the systematic (numerical and automatic) modification of the convex hull instead of developing new LMI equations for feasible controller design (this is the widely adopted approach). It is worth noting that both the TP model transformation and the LMI-based control design methods are numerically executable one after the other, and this makes the resolution of a wide class of problems possible in a straightforward and tractable, numerical way.
• The TP model transformation is capable of performing trade-off between complexity and accuracy of TP functions via discarding the higher-order singular values, in the same manner as the tensor HOSVD is used for complexity reduction.

References

Baranyi, P. (2018). Extension of the Multi-TP Model Transformation to Functions with Different Numbers of Variables. Complexity, 2018.

External links

• TPtoolBoxMATLAB