Gram matrix

Short description: Matrix of inner products of a set of vectors

In linear algebra, the Gram matrix (or Gramian matrix, Gramian) of a set of vectors [math]\displaystyle{ v_1,\dots, v_n }[/math] in an inner product space is the Hermitian matrix of inner products, whose entries are given by the inner product [math]\displaystyle{ G_{ij} = \left\langle v_i, v_j \right\rangle }[/math].^[1] If the vectors [math]\displaystyle{ v_1,\dots, v_n }[/math] are the columns of matrix [math]\displaystyle{ X }[/math] then the Gram matrix is [math]\displaystyle{ X^\dagger X }[/math] in the general case that the vector coordinates are complex numbers, which simplifies to [math]\displaystyle{ X^\top X }[/math] for the case that the vector coordinates are real numbers.

An important application is to compute linear independence: a set of vectors are linearly independent if and only if the Gram determinant (the determinant of the Gram matrix) is non-zero.

It is named after Jørgen Pedersen Gram.

Examples

For finite-dimensional real vectors in [math]\displaystyle{ \mathbb{R}^n }[/math] with the usual Euclidean dot product, the Gram matrix is [math]\displaystyle{ G = V^\top V }[/math], where [math]\displaystyle{ V }[/math] is a matrix whose columns are the vectors [math]\displaystyle{ v_k }[/math] and [math]\displaystyle{ V^\top }[/math] is its transpose whose rows are the vectors [math]\displaystyle{ v_k^\top }[/math]. For complex vectors in [math]\displaystyle{ \mathbb{C}^n }[/math], [math]\displaystyle{ G = V^\dagger V }[/math], where [math]\displaystyle{ V^\dagger }[/math] is the conjugate transpose of [math]\displaystyle{ V }[/math].

Given square-integrable functions [math]\displaystyle{ \{\ell_i(\cdot),\, i = 1,\dots,n\} }[/math] on the interval [math]\displaystyle{ \left[t_0, t_f\right] }[/math], the Gram matrix [math]\displaystyle{ G = \left[G_{ij}\right] }[/math] is:

[math]\displaystyle{ G_{ij} = \int_{t_0}^{t_f} \ell_i^*(\tau)\ell_j(\tau)\, d\tau. }[/math]

where [math]\displaystyle{ \ell_i^*(\tau) }[/math] is the complex conjugate of [math]\displaystyle{ \ell_i(\tau) }[/math].

For any bilinear form [math]\displaystyle{ B }[/math] on a finite-dimensional vector space over any field we can define a Gram matrix [math]\displaystyle{ G }[/math] attached to a set of vectors [math]\displaystyle{ v_1, \dots, v_n }[/math] by [math]\displaystyle{ G_{ij} = B\left(v_i, v_j\right) }[/math]. The matrix will be symmetric if the bilinear form [math]\displaystyle{ B }[/math] is symmetric.

Applications

In Riemannian geometry, given an embedded [math]\displaystyle{ k }[/math]-dimensional Riemannian manifold [math]\displaystyle{ M\subset \mathbb{R}^n }[/math] and a parametrization [math]\displaystyle{ \phi: U\to M }[/math] for [math]\displaystyle{ (x_1, \ldots, x_k)\in U\subset\mathbb{R}^k }[/math], the volume form [math]\displaystyle{ \omega }[/math] on [math]\displaystyle{ M }[/math] induced by the embedding may be computed using the Gramian of the coordinate tangent vectors: [math]\displaystyle{ \omega = \sqrt{\det G}\ dx_1 \cdots dx_k,\quad G = \left[\left\langle \frac{\partial\phi}{\partial x_i},\frac{\partial\phi}{\partial x_j}\right\rangle\right]. }[/math] This generalizes the classical surface integral of a parametrized surface [math]\displaystyle{ \phi:U\to S\subset \mathbb{R}^3 }[/math] for [math]\displaystyle{ (x, y)\in U\subset\mathbb{R}^2 }[/math]: [math]\displaystyle{ \int_S f\ dA = \iint_U f(\phi(x, y))\, \left|\frac{\partial\phi}{\partial x}\,{\times}\,\frac{\partial\phi}{\partial y}\right|\, dx\, dy. }[/math]
If the vectors are centered random variables, the Gramian is approximately proportional to the covariance matrix, with the scaling determined by the number of elements in the vector.
In quantum chemistry, the Gram matrix of a set of basis vectors is the overlap matrix.
In control theory (or more generally systems theory), the controllability Gramian and observability Gramian determine properties of a linear system.
Gramian matrices arise in covariance structure model fitting (see e.g., Jamshidian and Bentler, 1993, Applied Psychological Measurement, Volume 18, pp. 79–94).
In the finite element method, the Gram matrix arises from approximating a function from a finite dimensional space; the Gram matrix entries are then the inner products of the basis functions of the finite dimensional subspace.
In machine learning, kernel functions are often represented as Gram matrices.^[2] (Also see kernel PCA)
Since the Gram matrix over the reals is a symmetric matrix, it is diagonalizable and its eigenvalues are non-negative. The diagonalization of the Gram matrix is the singular value decomposition.

Properties

Positive-semidefiniteness

The Gram matrix is symmetric in the case the real product is real-valued; it is Hermitian in the general, complex case by definition of an inner product.

The Gram matrix is positive semidefinite, and every positive semidefinite matrix is the Gramian matrix for some set of vectors. The fact that the Gramian matrix is positive-semidefinite can be seen from the following simple derivation:

[math]\displaystyle{ x^\dagger \mathbf{G} x = \sum_{i,j}x_i^* x_j\left\langle v_i, v_j \right\rangle = \sum_{i,j}\left\langle x_i v_i, x_j v_j \right\rangle = \biggl\langle \sum_i x_i v_i, \sum_j x_j v_j \biggr\rangle = \biggl\| \sum_i x_i v_i \biggr\|^2 \geq 0 . }[/math]

The first equality follows from the definition of matrix multiplication, the second and third from the bi-linearity of the inner-product, and the last from the positive definiteness of the inner product. Note that this also shows that the Gramian matrix is positive definite if and only if the vectors [math]\displaystyle{ v_i }[/math] are linearly independent (that is, [math]\displaystyle{ \sum_i x_i v_i \neq 0 }[/math] for all [math]\displaystyle{ x }[/math]).^[1]

Finding a vector realization

Given any positive semidefinite matrix [math]\displaystyle{ M }[/math], one can decompose it as:

[math]\displaystyle{ M = B^\dagger B }[/math],

where [math]\displaystyle{ B^\dagger }[/math] is the conjugate transpose of [math]\displaystyle{ B }[/math] (or [math]\displaystyle{ M = B^\textsf{T} B }[/math] in the real case).

Here [math]\displaystyle{ B }[/math] is a [math]\displaystyle{ k \times n }[/math] matrix, where [math]\displaystyle{ k }[/math] is the rank of [math]\displaystyle{ M }[/math]. Various ways to obtain such a decomposition include computing the Cholesky decomposition or taking the non-negative square root of [math]\displaystyle{ M }[/math].

The columns [math]\displaystyle{ b^{(1)}, \dots, b^{(n)} }[/math] of [math]\displaystyle{ B }[/math] can be seen as n vectors in [math]\displaystyle{ \mathbb{C}^k }[/math] (or k-dimensional Euclidean space [math]\displaystyle{ \mathbb{R}^k }[/math], in the real case). Then

[math]\displaystyle{ M_{ij} = b^{(i)} \cdot b^{(j)} }[/math]

where the dot product [math]\displaystyle{ a \cdot b = \sum_{\ell=1}^k a_\ell^* b_\ell }[/math] is the usual inner product on [math]\displaystyle{ \mathbb{C}^k }[/math].

Thus a Hermitian matrix [math]\displaystyle{ M }[/math] is positive semidefinite if and only if it is the Gram matrix of some vectors [math]\displaystyle{ b^{(1)}, \dots, b^{(n)} }[/math]. Such vectors are called a vector realization of [math]\displaystyle{ M }[/math]. The infinite-dimensional analog of this statement is Mercer's theorem.

Uniqueness of vector realizations

If [math]\displaystyle{ M }[/math] is the Gram matrix of vectors [math]\displaystyle{ v_1,\dots,v_n }[/math] in [math]\displaystyle{ \mathbb{R}^k }[/math] then applying any rotation or reflection of [math]\displaystyle{ \mathbb{R}^k }[/math] (any orthogonal transformation, that is, any Euclidean isometry preserving 0) to the sequence of vectors results in the same Gram matrix. That is, for any [math]\displaystyle{ k \times k }[/math] orthogonal matrix [math]\displaystyle{ Q }[/math], the Gram matrix of [math]\displaystyle{ Q v_1,\dots, Q v_n }[/math] is also [math]\displaystyle{ M }[/math].

This is the only way in which two real vector realizations of [math]\displaystyle{ M }[/math] can differ: the vectors [math]\displaystyle{ v_1,\dots,v_n }[/math] are unique up to orthogonal transformations. In other words, the dot products [math]\displaystyle{ v_i \cdot v_j }[/math] and [math]\displaystyle{ w_i \cdot w_j }[/math] are equal if and only if some rigid transformation of [math]\displaystyle{ \mathbb{R}^k }[/math] transforms the vectors [math]\displaystyle{ v_1,\dots,v_n }[/math] to [math]\displaystyle{ w_1, \dots, w_n }[/math] and 0 to 0.

The same holds in the complex case, with unitary transformations in place of orthogonal ones. That is, if the Gram matrix of vectors [math]\displaystyle{ v_1, \dots, v_n }[/math] is equal to the Gram matrix of vectors [math]\displaystyle{ w_1, \dots, w_n }[/math] in [math]\displaystyle{ \mathbb{C}^k }[/math] then there is a unitary [math]\displaystyle{ k \times k }[/math] matrix [math]\displaystyle{ U }[/math] (meaning [math]\displaystyle{ U^\dagger U = I }[/math]) such that [math]\displaystyle{ v_i = U w_i }[/math] for [math]\displaystyle{ i = 1, \dots, n }[/math].^[3]

Other properties

Because [math]\displaystyle{ G = G^\dagger }[/math], it is necessarily the case that [math]\displaystyle{ G }[/math] and [math]\displaystyle{ G^\dagger }[/math] commute. That is, a real or complex Gram matrix [math]\displaystyle{ G }[/math] is also a normal matrix.
The Gram matrix of any orthonormal basis is the identity matrix. Equivalently, the Gram matrix of the rows or the columns of a real rotation matrix is the identity matrix. Likewise, the Gram matrix of the rows or columns of a unitary matrix is the identity matrix.
The rank of the Gram matrix of vectors in [math]\displaystyle{ \mathbb{R}^k }[/math] or [math]\displaystyle{ \mathbb{C}^k }[/math] equals the dimension of the space spanned by these vectors.^[1]

Gram determinant

The Gram determinant or Gramian is the determinant of the Gram matrix: [math]\displaystyle{ \bigl|G(\{v_1, \dots, v_n\})\bigr| = \begin{vmatrix} \langle v_1,v_1\rangle & \langle v_1,v_2\rangle &\dots & \langle v_1,v_n\rangle \\ \langle v_2,v_1\rangle & \langle v_2,v_2\rangle &\dots & \langle v_2,v_n\rangle \\ \vdots & \vdots & \ddots & \vdots \\ \langle v_n,v_1\rangle & \langle v_n,v_2\rangle &\dots & \langle v_n,v_n\rangle \end{vmatrix}. }[/math]

If [math]\displaystyle{ v_1, \dots, v_n }[/math] are vectors in [math]\displaystyle{ \mathbb{R}^m }[/math] then it is the square of the n-dimensional volume of the parallelotope formed by the vectors. In particular, the vectors are linearly independent if and only if the parallelotope has nonzero n-dimensional volume, if and only if Gram determinant is nonzero, if and only if the Gram matrix is nonsingular. When n > m the determinant and volume are zero. When n = m, this reduces to the standard theorem that the absolute value of the determinant of n n-dimensional vectors is the n-dimensional volume. The Gram determinant is also useful for computing the volume of the simplex formed by the vectors; its volume is $Volume(parallelotope) / n!$ .

The Gram determinant can also be expressed in terms of the exterior product of vectors by

[math]\displaystyle{ \bigl|G(v_1, \dots, v_n)\bigr| = \| v_1 \wedge \cdots \wedge v_n\|^2. }[/math]

When the vectors [math]\displaystyle{ v_1, \ldots, v_n \in \mathbb{R}^m }[/math] are defined from the positions of points [math]\displaystyle{ p_1, \ldots, p_n }[/math] relative to some reference point [math]\displaystyle{ p_{n+1} }[/math],

[math]\displaystyle{ (v_1, v_2, \ldots, v_n) = (p_1 - p_{n+1}, p_2 - p_{n+1}, \ldots, p_n - p_{n+1})\,, }[/math]

then the Gram determinant can be written as the difference of two Gram determinants,

[math]\displaystyle{ \bigl|G(\{v_1, \dots, v_n\})\bigr| = \bigl|G(\{(p_1, 1), \dots, (p_{n+1}, 1)\})\bigr| - \bigl|G(\{p_1, \dots, p_{n+1}\})\bigr|\,, }[/math]

where each [math]\displaystyle{ (p_j, 1) }[/math] is the corresponding point [math]\displaystyle{ p_j }[/math] supplemented with the coordinate value of 1 for an [math]\displaystyle{ (m+1) }[/math]-st dimension.^{[citation needed]} Note that in the common case that $n = m$ , the second term on the right-hand side will be zero.

Constructing an orthonormal basis

Given a set of linearly independent vectors [math]\displaystyle{ \{v_i\} }[/math] with Gram matrix [math]\displaystyle{ G }[/math] defined by [math]\displaystyle{ G_{ij}:= \langle v_i,v_j\rangle }[/math], one can construct an orthonormal basis

[math]\displaystyle{ u_i := \sum_j \bigl(G^{-1/2}\bigr)_{ji} v_j. }[/math]

In matrix notation, [math]\displaystyle{ U = V G^{-1/2} }[/math], where [math]\displaystyle{ U }[/math] has orthonormal basis vectors [math]\displaystyle{ \{u_i\} }[/math] and the matrix [math]\displaystyle{ V }[/math] is composed of the given column vectors [math]\displaystyle{ \{v_i\} }[/math].

The matrix [math]\displaystyle{ G^{-1/2} }[/math] is guaranteed to exist. Indeed, [math]\displaystyle{ G }[/math] is Hermitian, and so can be decomposed as [math]\displaystyle{ G=UDU^\dagger }[/math] with [math]\displaystyle{ U }[/math] a unitary matrix and [math]\displaystyle{ D }[/math] a real diagonal matrix. Additionally, the [math]\displaystyle{ v_i }[/math] are linearly independent if and only if [math]\displaystyle{ G }[/math] is positive definite, which implies that the diagonal entries of [math]\displaystyle{ D }[/math] are positive. [math]\displaystyle{ G^{-1/2} }[/math] is therefore uniquely defined by [math]\displaystyle{ G^{-1/2}:=UD^{-1/2}U^\dagger }[/math]. One can check that these new vectors are orthonormal:

[math]\displaystyle{ \begin{align} \langle u_i,u_j \rangle &= \sum_{i'} \sum_{j'} \Bigl\langle \bigl(G^{-1/2}\bigr)_{i'i} v_{i'},\bigl(G^{-1/2}\bigr)_{j'j} v_{j'} \Bigr\rangle \\[10mu] &= \sum_{i'} \sum_{j'} \bigl(G^{-1/2}\bigr)_{ii'} G_{i'j'} \bigl(G^{-1/2}\bigr)_{j'j} \\[8mu] &= \bigl(G^{-1/2} G G^{-1/2}\bigr)_{ij} = \delta_{ij} \end{align} }[/math]

where we used [math]\displaystyle{ \bigl(G^{-1/2}\bigr)^\dagger=G^{-1/2} }[/math].

References

↑ ^1.0 ^1.1 ^1.2 Horn & Johnson 2013, p. 441, p.441, Theorem 7.2.10
↑ Lanckriet, G. R. G.; Cristianini, N.; Bartlett, P.; Ghaoui, L. E.; Jordan, M. I. (2004). "Learning the kernel matrix with semidefinite programming". Journal of Machine Learning Research 5: 27–72 [p. 29]. https://dl.acm.org/citation.cfm?id=894170.
↑ (Horn Johnson), p. 452, Theorem 7.3.11

Horn, Roger A.; Johnson, Charles R. (2013). Matrix Analysis (2nd ed.). Cambridge University Press. ISBN 978-0-521-54823-6.

External links

Hazewinkel, Michiel, ed. (2001), "Gram matrix", Encyclopedia of Mathematics, Springer Science+Business Media B.V. / Kluwer Academic Publishers, ISBN 978-1-55608-010-4, https://www.encyclopediaofmath.org/index.php?title=p/g044750
Volumes of parallelograms by Frank Jones

fr:Matrice de Gram

0.00

(0 votes)

Original source: https://en.wikipedia.org/wiki/Gram matrix. Read more

[HJ-7.2.10-1] 1.0 ^1.1 ^1.2 Horn & Johnson 2013, p. 441, p.441, Theorem 7.2.10

[2] Lanckriet, G. R. G.; Cristianini, N.; Bartlett, P.; Ghaoui, L. E.; Jordan, M. I. (2004). "Learning the kernel matrix with semidefinite programming". Journal of Machine Learning Research 5: 27–72 [p. 29]. https://dl.acm.org/citation.cfm?id=894170.

[3] (Horn Johnson), p. 452, Theorem 7.3.11

[1]

[2]

[3]

v t e Matrix classes
Explicitly constrained entries	(0,1) Alternant Anti-diagonal Anti-Hermitian Anti-symmetric Arrowhead Band Bidiagonal Binary Bisymmetric Block-diagonal Block Block tridiagonal Boolean Cauchy Centrosymmetric Conference Complex Hadamard Copositive Diagonally dominant Diagonal Discrete Fourier Transform Elementary Equivalent Frobenius Generalized permutation Hadamard Hankel Hermitian Hessenberg Hollow Integer Logical Markov Metzler Monomial Moore Nonnegative Partitioned Parisi Pentadiagonal Permutation Persymmetric Polynomial Positive Quaternionic Sign Signature Skew-Hermitian Skew-symmetric Skyline Sparse Sylvester Symmetric Toeplitz Triangular Tridiagonal Unitary Vandermonde Walsh Z
Constant	Exchange Hilbert Identity Lehmer Of ones Pascal Pauli Redheffer Shift Zero
Conditions on eigenvalues or eigenvectors	Companion Convergent Defective Diagonalizable Hurwitz Positive-definite Stability Stieltjes
Satisfying conditions on products or inverses	Congruent Idempotent or Projection Invertible Involutory Nilpotent Normal Orthogonal Orthonormal Singular Unimodular Unipotent Totally unimodular Weighing
With specific applications	Adjugate Alternating sign Augmented Bézout Carleman Cartan Circulant Cofactor Commutation Confusion Coxeter Derogatory Distance Duplication Elimination Euclidean distance Fundamental (linear differential equation) Generator Gramian Hessian Householder Jacobian Moment Payoff Pick Random Rotation Seifert Shear Similarity Symplectic Totally positive Transformation Wedderburn X–Y–Z
Used in statistics	Bernoulli Centering Correlation Covariance Design Dispersion Doubly stochastic Fisher information Hat Precision Stochastic Transition
Used in graph theory	Adjacency Biadjacency Degree Edmonds Incidence Laplacian Seidel adjacency Skew-adjacency Tutte
Used in science and engineering	Cabibbo–Kobayashi–Maskawa Density Fundamental (computer vision) Fuzzy associative Gamma Gell-Mann Hamiltonian Irregular Overlap S State transition Substitution Z (chemistry)
Related terms	Jordan canonical form Linear independence Matrix exponential Matrix representation of conic sections Perfect matrix Pseudoinverse Quaternionic matrix Row echelon form Wronskian
List of matrices Category:Matrices

Anonymous

Search

Gram matrix

Namespaces

More

Page actions

Contents

Examples

Applications

Properties

Positive-semidefiniteness

Finding a vector realization

Uniqueness of vector realizations

Other properties

Gram determinant

Constructing an orthonormal basis

See also

References

External links

Navigation

Navigation

Help

Translate

Wiki tools

Wiki tools

Anonymous

Search

Gram matrix

Examples

Applications

Properties

Positive-semidefiniteness

Finding a vector realization

Uniqueness of vector realizations

Other properties

Gram determinant

Constructing an orthonormal basis

See also

References

External links

Navigation

Wiki tools

Page tools

Other projects

Categories