Definite symmetric matrix

Not to be confused with Positive matrix and Totally positive matrix.

In linear algebra, a symmetric [math]\displaystyle{ n \times n }[/math] real matrix [math]\displaystyle{ M }[/math] is said to be positive-definite if the scalar [math]\displaystyle{ z^\textsf{T}Mz }[/math] is strictly positive for every non-zero column vector [math]\displaystyle{ z }[/math] of [math]\displaystyle{ n }[/math] real numbers. Here [math]\displaystyle{ z^\textsf{T} }[/math] denotes the transpose of [math]\displaystyle{ z }[/math]. When interpreting [math]\displaystyle{ Mz }[/math] as the output of an operator, [math]\displaystyle{ M }[/math], that is acting on an input, [math]\displaystyle{ z }[/math], the property of positive definiteness implies that the output always has a positive inner product with the input, as often observed in physical processes.

More generally, a complex [math]\displaystyle{ n\times n }[/math] Hermitian matrix [math]\displaystyle{ M }[/math] is said to be positive-definite if the scalar [math]\displaystyle{ z^* Mz }[/math] is strictly positive for every non-zero column vector [math]\displaystyle{ z }[/math] of [math]\displaystyle{ n }[/math] complex numbers. Here [math]\displaystyle{ z^* }[/math] denotes the conjugate transpose of [math]\displaystyle{ z }[/math]. Note that [math]\displaystyle{ z^* Mz }[/math] is automatically real since [math]\displaystyle{ M }[/math] is Hermitian.

Positive semi-definite matrices are defined similarly, except that the above scalars [math]\displaystyle{ z^\textsf{T}Mz }[/math] or [math]\displaystyle{ z^* Mz }[/math] must be positive or zero (i.e. non-negative). Negative-definite and negative semi-definite matrices are defined analogously. A matrix that is not positive semi-definite and not negative semi-definite is called indefinite.

The matrix [math]\displaystyle{ M }[/math] is positive-definite if and only if the bilinear form [math]\displaystyle{ \langle z, w\rangle = z^\textsf{T} Mw }[/math] is positive-definite (and similarly for a positive-definite sesquilinear form in the complex case). This is a coordinate realization of an inner product on a vector space.

Some authors use more general definitions of definiteness, including some non-symmetric real matrices, or non-Hermitian complex ones.

Definitions

In the following definitions, [math]\displaystyle{ x^\textsf{T} }[/math] is the transpose of [math]\displaystyle{ x }[/math], [math]\displaystyle{ x^* }[/math] is the conjugate transpose of [math]\displaystyle{ x }[/math] and [math]\displaystyle{ \mathbf{0} }[/math] denotes the n-dimensional zero-vector.

Definitions for real matrices

An [math]\displaystyle{ n \times n }[/math] symmetric real matrix [math]\displaystyle{ M }[/math] is said to be positive-definite if [math]\displaystyle{ x^\textsf{T} Mx \gt 0 }[/math] for all non-zero [math]\displaystyle{ x }[/math] in [math]\displaystyle{ \mathbb{R}^n }[/math]. Formally,

[math]\displaystyle{ M \text{ positive-definite} \quad \iff \quad x^\textsf{T} Mx \gt 0 \text{ for all } x \in \mathbb{R}^n \setminus \mathbf{0} }[/math]

An [math]\displaystyle{ n \times n }[/math] symmetric real matrix [math]\displaystyle{ M }[/math] is said to be positive semidefinite or non-negative-definite if [math]\displaystyle{ x^\textsf{T} Mx \geq 0 }[/math] for all [math]\displaystyle{ x }[/math] in [math]\displaystyle{ \mathbb{R}^n }[/math]. Formally,

[math]\displaystyle{ M \text{ positive semi-definite} \quad \iff \quad x^\textsf{T} Mx \geq 0 \text{ for all } x \in \mathbb{R}^n }[/math]

An [math]\displaystyle{ n \times n }[/math] symmetric real matrix [math]\displaystyle{ M }[/math] is said to be negative-definite if [math]\displaystyle{ x^\textsf{T} Mx \lt 0 }[/math] for all non-zero [math]\displaystyle{ x }[/math] in [math]\displaystyle{ \mathbb{R}^n }[/math]. Formally,

[math]\displaystyle{ M \text{ negative-definite} \quad \iff \quad x^\textsf{T} Mx \lt 0 \text{ for all } x \in \mathbb{R}^n \setminus \mathbf{0} }[/math]

An [math]\displaystyle{ n \times n }[/math] symmetric real matrix [math]\displaystyle{ M }[/math] is said to be negative-semidefinite or non-positive-definite if [math]\displaystyle{ x^\textsf{T} Mx \leq 0 }[/math] for all [math]\displaystyle{ x }[/math] in [math]\displaystyle{ \mathbb{R}^n }[/math]. Formally,

[math]\displaystyle{ M \text{ negative semi-definite} \quad \iff \quad x^\textsf{T} Mx \leq 0 \text{ for all } x \in \mathbb{R}^n }[/math]

An [math]\displaystyle{ n \times n }[/math] symmetric real matrix which is neither positive semidefinite nor negative semidefinite is called indefinite.

Definitions for complex matrices

The following definitions all involve the term [math]\displaystyle{ x^* Mx }[/math]. Notice that this is always a real number for any Hermitian square matrix [math]\displaystyle{ M }[/math].

An [math]\displaystyle{ n \times n }[/math] Hermitian complex matrix [math]\displaystyle{ M }[/math] is said to be positive-definite if [math]\displaystyle{ x^* Mx \gt 0 }[/math] for all non-zero [math]\displaystyle{ x }[/math] in [math]\displaystyle{ \mathbb{C}^n }[/math]. Formally,

[math]\displaystyle{ M \text{ positive-definite} \quad \iff \quad x^* Mx \gt 0 \text{ for all } x \in \mathbb{C}^n \setminus \mathbf{0} }[/math]

An [math]\displaystyle{ n \times n }[/math] Hermitian complex matrix [math]\displaystyle{ M }[/math] is said to be positive semi-definite or non-negative-definite if [math]\displaystyle{ x^* Mx \geq 0 }[/math] for all [math]\displaystyle{ x }[/math] in [math]\displaystyle{ \mathbb{C}^n }[/math]. Formally,

[math]\displaystyle{ M \text{ positive semi-definite} \quad \iff \quad x^* Mx \geq 0 \text{ for all } x \in \mathbb{C}^n }[/math]

An [math]\displaystyle{ n \times n }[/math] Hermitian complex matrix [math]\displaystyle{ M }[/math] is said to be negative-definite if [math]\displaystyle{ x^* Mx \lt 0 }[/math] for all non-zero [math]\displaystyle{ x }[/math] in [math]\displaystyle{ \mathbb{C}^n }[/math]. Formally,

[math]\displaystyle{ M \text{ negative-definite} \quad \iff \quad x^* Mx \lt 0 \text{ for all } x \in \mathbb{C}^n \setminus \mathbf{0} }[/math]

An [math]\displaystyle{ n \times n }[/math] Hermitian complex matrix [math]\displaystyle{ M }[/math] is said to be negative semi-definite or non-positive-definite if [math]\displaystyle{ x^* Mx \leq 0 }[/math] for all [math]\displaystyle{ x }[/math] in [math]\displaystyle{ \mathbb{C}^n }[/math]. Formally,

[math]\displaystyle{ M \text{ negative semi-definite} \quad \iff \quad x^* Mx \leq 0 \text{ for all } x \in \mathbb{C}^n }[/math]

An [math]\displaystyle{ n \times n }[/math] Hermitian complex matrix which is neither positive semidefinite nor negative semidefinite is called indefinite.

Consistency between real and complex definitions

Since every real matrix is also a complex matrix, the definitions of "definiteness" for the two classes must agree.

For complex matrices, the most common definition says that "[math]\displaystyle{ M }[/math] is positive-definite if and only if [math]\displaystyle{ z^* Mz }[/math] is real and positive for all non-zero complex column vectors [math]\displaystyle{ z }[/math]". This condition implies that [math]\displaystyle{ M }[/math] is Hermitian (i.e. its transpose is equal to its conjugate). To see this, consider the matrices [math]\displaystyle{ A = \tfrac{1}{2} \left(M + M^*\right) }[/math] and [math]\displaystyle{ B = \tfrac{1}{2i} \left(M - M^*\right) }[/math], so that [math]\displaystyle{ M = A + iB }[/math] and [math]\displaystyle{ z^* Mz = z^* Az + iz^* Bz }[/math]. The matrices [math]\displaystyle{ A }[/math] and [math]\displaystyle{ B }[/math] are Hermitian, therefore [math]\displaystyle{ z^* Az }[/math] and [math]\displaystyle{ z^* Bz }[/math] are individually real. If [math]\displaystyle{ z^* Mz }[/math] is real, then [math]\displaystyle{ z^* Bz }[/math] must be zero for all [math]\displaystyle{ z }[/math]. Then [math]\displaystyle{ B }[/math] is the zero matrix and [math]\displaystyle{ M = A }[/math], proving that [math]\displaystyle{ M }[/math] is Hermitian.

By this definition, a positive-definite real matrix [math]\displaystyle{ M }[/math] is Hermitian, hence symmetric; and [math]\displaystyle{ z^\textsf{T} Mz }[/math] is positive for all non-zero real column vectors [math]\displaystyle{ z }[/math]. However the last condition alone is not sufficient for [math]\displaystyle{ M }[/math] to be positive-definite. For example, if

[math]\displaystyle{ M = \begin{bmatrix} 1 & 1 \\ -1 & 1 \end{bmatrix}, }[/math]

then for any real vector [math]\displaystyle{ z }[/math] with entries [math]\displaystyle{ a }[/math] and [math]\displaystyle{ b }[/math] we have [math]\displaystyle{ z^\textsf{T} Mz = (a + b)a + (-a + b)b = a^2 + b^2 }[/math], which is always positive if [math]\displaystyle{ z }[/math] is not zero. However, if [math]\displaystyle{ z }[/math] is the complex vector with entries [math]\displaystyle{ 1 }[/math] and [math]\displaystyle{ i }[/math], one gets

[math]\displaystyle{ z^* M z = [1, -i] M [1, i]^\textsf{T} = [1 + i, 1 - i] [1, i]^\textsf{T} = 2+2i }[/math]

which is not real. Therefore, [math]\displaystyle{ M }[/math] is not positive-definite.

On the other hand, for a symmetric real matrix [math]\displaystyle{ M }[/math], the condition "[math]\displaystyle{ z^\textsf{T} Mz \gt 0 }[/math] for all nonzero real vectors [math]\displaystyle{ z }[/math]" does imply that [math]\displaystyle{ M }[/math] is positive-definite in the complex sense.

Notation

If a Hermitian matrix [math]\displaystyle{ M }[/math] is positive semi-definite, one sometimes writes [math]\displaystyle{ M \succeq 0 }[/math] and if [math]\displaystyle{ M }[/math] is positive-definite one writes [math]\displaystyle{ M \succ 0 }[/math]. To denote that [math]\displaystyle{ M }[/math] is negative semi-definite one writes [math]\displaystyle{ M \preceq 0 }[/math] and to denote that [math]\displaystyle{ M }[/math] is negative-definite one writes [math]\displaystyle{ M \prec 0 }[/math].

The notion comes from functional analysis where positive semidefinite matrices define positive operators.

A common alternative notation is [math]\displaystyle{ M \geq 0 }[/math], [math]\displaystyle{ M \gt 0 }[/math], [math]\displaystyle{ M \leq 0 }[/math] and [math]\displaystyle{ M \lt 0 }[/math] for positive semi-definite and positive-definite, negative semi-definite and negative-definite matrices, respectively. This may be confusing, as sometimes nonnegative matrices (respectively, nonpositive matrices) are also denoted in this way.

Examples

• The identity matrix [math]\displaystyle{ I = \begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix} }[/math] is positive-definite (and as such also positive semi-definite). It is a real symmetric matrix, and, for any non-zero column vector z with real entries a and b, one has

  [math]\displaystyle{ z^\textsf{T}Iz = \begin{bmatrix} a & b \end{bmatrix} \begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix} \begin{bmatrix} a \\ b \end{bmatrix} = a^2 + b^2 }[/math].

  Seen as a complex matrix, for any non-zero column vector z with complex entries a and b one has

  [math]\displaystyle{ z^*Iz = \begin{bmatrix} \overline{a} & \overline{b} \end{bmatrix} \begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix} \begin{bmatrix} a \\ b\end{bmatrix} = \overline{a}a + \overline{b}b = |a|^2 + |b|^2 }[/math].

  Either way, the result is positive since [math]\displaystyle{ z }[/math] is not the zero vector (that is, at least one of [math]\displaystyle{ a }[/math] and [math]\displaystyle{ b }[/math] is not zero).
• The real symmetric matrix

  [math]\displaystyle{ M = \begin{bmatrix} 2 & -1 & 0 \\ -1 & 2 & -1 \\ 0 & -1 & 2 \end{bmatrix} }[/math]

  is positive-definite since for any non-zero column vector z with entries a, b and c, we have

  [math]\displaystyle{ \begin{align} z^\textsf{T} Mz = \left(z^\textsf{T}M\right) z &= \begin{bmatrix} (2a - b) & (-a + 2b - c) & (-b + 2c) \end{bmatrix} \begin{bmatrix} a \\ b \\ c \end{bmatrix} \\ &= (2a - b)a + (-a + 2b - c)b + (-b + 2c)c \\ &= 2a^2 - ba - ab + 2b^2 - cb - bc + 2c^2 \\ &= 2a^2 - 2ab + 2b^2 - 2bc + 2c^2 \\ &= a^2 + a^2 - 2ab + b^2 + b^2- 2bc + c^2 + c^2 \\ &= a^2 + (a - b)^2 + (b - c)^2 + c^2 \end{align} }[/math]

  This result is a sum of squares, and therefore non-negative; and is zero only if [math]\displaystyle{ a = b = c = 0 }[/math], that is, when z is the zero vector.
• For any real invertible matrix [math]\displaystyle{ A }[/math], the product [math]\displaystyle{ A^\textsf{T}A }[/math] is a positive definite matrix. A simple proof is that for any non-zero vector [math]\displaystyle{ z }[/math], the condition [math]\displaystyle{ z^\textsf{T} A^\textsf{T} Az = (Az)^\textsf{T}(Az) = \|Az\|^2 \gt 0, }[/math] since the invertibility of matrix [math]\displaystyle{ A }[/math] means that [math]\displaystyle{ Az \neq 0. }[/math]
• The example [math]\displaystyle{ M }[/math] above shows that a matrix in which some elements are negative may still be positive definite. Conversely, a matrix whose entries are all positive is not necessarily positive definite, as for example

  [math]\displaystyle{ N = \begin{bmatrix} 1 & 2 \\ 2 & 1 \end{bmatrix}, }[/math]

  for which [math]\displaystyle{ \begin{bmatrix} -1 & 1 \end{bmatrix}N\begin{bmatrix} -1 & 1 \end{bmatrix}^\textsf{T} = -2 \lt 0. }[/math]

Eigenvalues

Let [math]\displaystyle{ M }[/math] be an [math]\displaystyle{ n \times n }[/math] Hermitian matrix. This implies all its eigenvalues are real.

• [math]\displaystyle{ M }[/math] is positive definite if and only if all of its eigenvalues are positive.
• [math]\displaystyle{ M }[/math] is positive semi-definite if and only if all of its eigenvalues are non-negative.
• [math]\displaystyle{ M }[/math] is negative definite if and only if all of its eigenvalues are negative
• [math]\displaystyle{ M }[/math] is negative semi-definite if and only if all of its eigenvalues are non-positive.
• [math]\displaystyle{ M }[/math] is indefinite if and only if it has both positive and negative eigenvalues.

Let [math]\displaystyle{ P^{-1} DP }[/math] be an eigendecomposition of [math]\displaystyle{ M }[/math], where [math]\displaystyle{ P }[/math] is a unitary complex matrix whose rows comprise an orthonormal basis of eigenvectors of [math]\displaystyle{ M }[/math], and [math]\displaystyle{ D }[/math] is a real diagonal matrix whose main diagonal contains the corresponding eigenvalues. The matrix [math]\displaystyle{ M }[/math] may be regarded as a diagonal matrix [math]\displaystyle{ D }[/math] that has been re-expressed in coordinates of the basis [math]\displaystyle{ P }[/math]. In particular, the one-to-one change of variable [math]\displaystyle{ y = Pz }[/math] shows that [math]\displaystyle{ z^* Mz }[/math] is real and positive for any complex vector [math]\displaystyle{ z }[/math] if and only if [math]\displaystyle{ y^* Dy }[/math] is real and positive for any [math]\displaystyle{ y }[/math]; in other words, if [math]\displaystyle{ D }[/math] is positive definite. For a diagonal matrix, this is true only if each element of the main diagonal—that is, every eigenvalue of [math]\displaystyle{ M }[/math]—is positive. Since the spectral theorem guarantees all eigenvalues of a Hermitian matrix to be real, the positivity of eigenvalues can be checked using Descartes' rule of alternating signs when the characteristic polynomial of a real, symmetric matrix [math]\displaystyle{ M }[/math] is available.

Decomposition

Let [math]\displaystyle{ M }[/math] be an [math]\displaystyle{ n \times n }[/math] Hermitian matrix. [math]\displaystyle{ M }[/math] is positive semidefinite if and only if it can be decomposed as a product

[math]\displaystyle{ M = B^* B }[/math]

of a matrix [math]\displaystyle{ B }[/math] with its conjugate transpose.

When [math]\displaystyle{ M }[/math] is real, [math]\displaystyle{ B }[/math] can be real as well and the decomposition can be written as

[math]\displaystyle{ M = B^\textsf{T} B. }[/math]

[math]\displaystyle{ M }[/math] is positive definite if and only if such a decomposition exists with [math]\displaystyle{ B }[/math] invertible. More generally, [math]\displaystyle{ M }[/math] is positive semidefinite with rank [math]\displaystyle{ k }[/math] if and only if a decomposition exists with a [math]\displaystyle{ k \times n }[/math] matrix [math]\displaystyle{ B }[/math] of full row rank (i.e. of rank [math]\displaystyle{ k }[/math]). Moreover, for any decomposition [math]\displaystyle{ M = B^* B }[/math], [math]\displaystyle{ \operatorname{rank}(M) = \operatorname{rank}(B) }[/math].

Proof

If [math]\displaystyle{ M = B^* B }[/math], then [math]\displaystyle{ x^* M x = (x^* B^*) (B x) = \|B x \|^2 \geq 0 }[/math], so [math]\displaystyle{ M }[/math] is positive semidefinite. If moreover [math]\displaystyle{ B }[/math] is invertible then the inequality is strict for [math]\displaystyle{ x \neq 0 }[/math], so [math]\displaystyle{ M }[/math] is positive definite. If [math]\displaystyle{ B }[/math] is [math]\displaystyle{ k \times n }[/math] of rank [math]\displaystyle{ k }[/math], then [math]\displaystyle{ \operatorname{rank}(M) = \operatorname{rank}(B^*) = k }[/math]. In the other direction, suppose [math]\displaystyle{ M }[/math] is positive semidefinite. Since [math]\displaystyle{ M }[/math] is Hermitian, it has an eigendecomposition [math]\displaystyle{ M=Q^{-1} D Q }[/math] where [math]\displaystyle{ Q }[/math] is unitary and [math]\displaystyle{ D }[/math] is a diagonal matrix whose entries are the eigenvalues of [math]\displaystyle{ M }[/math] Since [math]\displaystyle{ M }[/math] is positive semidefinite, the eigenvalues are non-negative real numbers, so one can define [math]\displaystyle{ D^{\frac{1}{2}} }[/math] as the diagonal matrix whose entries are non-negative square roots of eigenvalues. Then [math]\displaystyle{ M=Q^{-1} D Q = Q^* D Q = Q^* D^{\frac{1}{2}} D^{\frac{1}{2}} Q = Q^* D^{\frac{1}{2}*} D^{\frac{1}{2}} Q = B^* B }[/math] for [math]\displaystyle{ B = D^{\frac{1}{2}} Q }[/math]. If moreover [math]\displaystyle{ M }[/math] is positive definite, then the eigenvalues are (strictly) positive, so [math]\displaystyle{ D^{\frac{1}{2}} }[/math] is invertible, and hence [math]\displaystyle{ B = D^{\frac{1}{2}} Q }[/math] is invertible as well. If [math]\displaystyle{ M }[/math] has rank [math]\displaystyle{ k }[/math], then it has exactly [math]\displaystyle{ k }[/math] positive eigenvalues and the others are zero, hence in [math]\displaystyle{ B = D^{\frac{1}{2}} Q }[/math] all but [math]\displaystyle{ k }[/math] rows are all zeroed. Cutting the zero rows gives a [math]\displaystyle{ k \times n }[/math] matrix [math]\displaystyle{ B' }[/math] such that [math]\displaystyle{ B'^* B' = B^* B = M }[/math].

The columns [math]\displaystyle{ b_1,\dots,b_n }[/math] of [math]\displaystyle{ B }[/math] can be seen as vectors in the complex or real vector space [math]\displaystyle{ \mathbb{R}^k }[/math], respectively. Then the entries of [math]\displaystyle{ M }[/math] are inner products (that is dot products, in the real case) of these vectors

[math]\displaystyle{ M_{ij} = \langle b_i, b_j\rangle. }[/math]

In other words, a Hermitian matrix [math]\displaystyle{ M }[/math] is positive semidefinite if and only if it is the Gram matrix of some vectors [math]\displaystyle{ b_1,\dots,b_n }[/math]. It is positive definite if and only if it is the Gram matrix of some linearly independent vectors. In general, the rank of the Gram matrix of vectors [math]\displaystyle{ b_1,\dots,b_n }[/math] equals the dimension of the space spanned by these vectors.

Uniqueness up to unitary transformations

The decomposition is not unique: if [math]\displaystyle{ M = B^* B }[/math] for some [math]\displaystyle{ k \times n }[/math] matrix [math]\displaystyle{ B }[/math] and if [math]\displaystyle{ Q }[/math] is any unitary [math]\displaystyle{ k \times k }[/math] matrix (meaning [math]\displaystyle{ Q^* Q = Q Q^* = I }[/math]), then [math]\displaystyle{ M = B^* B = B^* Q^* Q B =A^* A }[/math] for [math]\displaystyle{ A=Q B }[/math].

However, this is the only way in which two decompositions can differ: the decomposition is unique up to unitary transformations. More formally, if [math]\displaystyle{ A }[/math] is a [math]\displaystyle{ k \times n }[/math] matrix and [math]\displaystyle{ B }[/math] is a [math]\displaystyle{ \ell \times n }[/math] matrix such that [math]\displaystyle{ A^* A = B^* B }[/math], then there is a [math]\displaystyle{ \ell \times k }[/math] matrix [math]\displaystyle{ Q }[/math] with orthonormal columns (meaning [math]\displaystyle{ Q^* Q = I_{k \times k} }[/math]) such that [math]\displaystyle{ B = Q A }[/math]. When [math]\displaystyle{ \ell = k }[/math] this means [math]\displaystyle{ Q }[/math] is unitary.

This statement has an intuitive geometric interpretation in the real case: let the columns of [math]\displaystyle{ A }[/math] and [math]\displaystyle{ B }[/math] be the vectors [math]\displaystyle{ a_1,\dots,a_n }[/math] and [math]\displaystyle{ b_1,\dots,b_n }[/math] in [math]\displaystyle{ \mathbb{R}^k }[/math]. A real unitary matrix is an orthogonal matrix, which describes a rigid transformation (an isometry of Euclidean space [math]\displaystyle{ \mathbb{R}^k }[/math]) preserving the 0 point (i.e. rotations and reflections, without translations). Therefore, the dot products [math]\displaystyle{ a_i \cdot a_j }[/math] and [math]\displaystyle{ b_i \cdot b_j }[/math] are equal if and only if some rigid transformation of [math]\displaystyle{ \mathbb{R}^k }[/math] transforms the vectors [math]\displaystyle{ a_1,\dots,a_n }[/math] to [math]\displaystyle{ b_1,\dots,b_n }[/math] (and 0 to 0).

Square root

A matrix [math]\displaystyle{ M }[/math] is positive semidefinite if and only if there is a positive semidefinite matrix [math]\displaystyle{ B }[/math] (in particular [math]\displaystyle{ B }[/math] is Hermitian, so [math]\displaystyle{ B^* = B }[/math]) satisfying [math]\displaystyle{ M = B B }[/math]. This matrix [math]\displaystyle{ B }[/math] is unique, is called the non-negative square root of [math]\displaystyle{ M }[/math], and is denoted with [math]\displaystyle{ B = M^\frac{1}{2} }[/math]. When [math]\displaystyle{ M }[/math] is positive definite, so is [math]\displaystyle{ M^\frac{1}{2} }[/math], hence it is also called the positive root of [math]\displaystyle{ M }[/math].

The non-negative square root should not be confused with other decompositions [math]\displaystyle{ M = B^*B }[/math]. Some authors use the name square root and [math]\displaystyle{ M^\frac{1}{2} }[/math] for any such decomposition, or specifically for the Cholesky decomposition, or any decomposition of the form [math]\displaystyle{ M = B B }[/math]; other only use it for the non-negative square root.

If [math]\displaystyle{ M \gt N \gt 0 }[/math] then [math]\displaystyle{ M^\frac{1}{2} \gt N^\frac{1}{2} \gt 0 }[/math].

Cholesky decomposition

A positive semidefinite matrix [math]\displaystyle{ M }[/math] can be written as [math]\displaystyle{ M = LL^* }[/math], where [math]\displaystyle{ L }[/math] is lower triangular with non-negative diagonal (equivalently [math]\displaystyle{ M = B^*B }[/math] where [math]\displaystyle{ B=L^* }[/math] is upper triangular); this is the Cholesky decomposition. If [math]\displaystyle{ M }[/math] is positive definite, then the diagonal of [math]\displaystyle{ L }[/math] is positive and the Cholesky decomposition is unique. The Cholesky decomposition is especially useful for efficient numerical calculations. A closely related decomposition is the LDL decomposition, [math]\displaystyle{ M = L D L^* }[/math], where [math]\displaystyle{ D }[/math] is diagonal and [math]\displaystyle{ L }[/math] is lower unitriangular.

Other characterizations

Let [math]\displaystyle{ M }[/math] be an [math]\displaystyle{ n \times n }[/math] Hermitian matrix. The following properties are equivalent to [math]\displaystyle{ M }[/math] being positive definite:

The associated sesquilinear form is an inner product

The sesquilinear form defined by [math]\displaystyle{ M }[/math] is the function [math]\displaystyle{ \langle \cdot, \cdot\rangle }[/math] from [math]\displaystyle{ \mathbb{C}^n \times \mathbb{C}^n }[/math] to [math]\displaystyle{ \mathbb{C}^n }[/math] such that [math]\displaystyle{ \langle x, y \rangle := y^*M x }[/math] for all [math]\displaystyle{ x }[/math] and [math]\displaystyle{ y }[/math] in [math]\displaystyle{ \mathbb{C}^n }[/math], where [math]\displaystyle{ y^* }[/math] is the conjugate transpose of [math]\displaystyle{ y }[/math]. For any complex matrix [math]\displaystyle{ M }[/math], this form is linear in [math]\displaystyle{ x }[/math] and semilinear in [math]\displaystyle{ y }[/math]. Therefore, the form is an inner product on [math]\displaystyle{ \mathbb{C}^n }[/math] if and only if [math]\displaystyle{ \langle z, z \rangle }[/math] is real and positive for all nonzero [math]\displaystyle{ z }[/math]; that is if and only if [math]\displaystyle{ M }[/math] is positive definite. (In fact, every inner product on [math]\displaystyle{ \mathbb{C}^n }[/math] arises in this fashion from a Hermitian positive definite matrix.)

Its leading principal minors are all positive

The kth leading principal minor of a matrix [math]\displaystyle{ M }[/math] is the determinant of its upper-left [math]\displaystyle{ k \times k }[/math] sub-matrix. It turns out that a matrix is positive definite if and only if all these determinants are positive. This condition is known as Sylvester's criterion, and provides an efficient test of positive definiteness of a symmetric real matrix. Namely, the matrix is reduced to an upper triangular matrix by using elementary row operations, as in the first part of the Gaussian elimination method, taking care to preserve the sign of its determinant during pivoting process. Since the kth leading principal minor of a triangular matrix is the product of its diagonal elements up to row [math]\displaystyle{ k }[/math], Sylvester's criterion is equivalent to checking whether its diagonal elements are all positive. This condition can be checked each time a new row [math]\displaystyle{ k }[/math] of the triangular matrix is obtained.

A positive semidefinite matrix is positive definite if and only if it is invertible. A matrix [math]\displaystyle{ M }[/math] is negative (semi)definite if and only if [math]\displaystyle{ -M }[/math] is positive (semi)definite.

Quadratic forms

The (purely) quadratic form associated with a real [math]\displaystyle{ n \times n }[/math] matrix [math]\displaystyle{ M }[/math] is the function [math]\displaystyle{ Q : \mathbb{R}^n \to \mathbb{R} }[/math] such that [math]\displaystyle{ Q(x) = x^\textsf{T} Mx }[/math] for all [math]\displaystyle{ x }[/math]. [math]\displaystyle{ M }[/math] can be assumed symmetric by replacing it with [math]\displaystyle{ \tfrac{1}{2} \left(M + M^\textsf{T}\right) }[/math].

A symmetric matrix [math]\displaystyle{ M }[/math] is positive definite if and only if its quadratic form is a strictly convex function.

More generally, any quadratic function from [math]\displaystyle{ \mathbb{R}^n }[/math] to [math]\displaystyle{ \mathbb{R} }[/math] can be written as [math]\displaystyle{ x^\textsf{T} Mx + x^\textsf{T} b + c }[/math] where [math]\displaystyle{ M }[/math] is a symmetric [math]\displaystyle{ n \times n }[/math] matrix, [math]\displaystyle{ b }[/math] is a real [math]\displaystyle{ n }[/math]-vector, and [math]\displaystyle{ c }[/math] a real constant. This quadratic function is strictly convex, and hence has a unique finite global minimum, if and only if [math]\displaystyle{ M }[/math] is positive definite. For this reason, positive definite matrices play an important role in optimization problems.

Simultaneous diagonalization

A symmetric matrix and another symmetric and positive definite matrix can be simultaneously diagonalized, although not necessarily via a similarity transformation. This result does not extend to the case of three or more matrices. In this section we write for the real case. Extension to the complex case is immediate.

Let [math]\displaystyle{ M }[/math] be a symmetric and [math]\displaystyle{ N }[/math] a symmetric and positive definite matrix. Write the generalized eigenvalue equation as [math]\displaystyle{ (M - \lambda N)x = 0 }[/math] where we impose that [math]\displaystyle{ x }[/math] be normalized, i.e. [math]\displaystyle{ x^\textsf{T} Nx = 1 }[/math]. Now we use Cholesky decomposition to write the inverse of [math]\displaystyle{ N }[/math] as [math]\displaystyle{ Q^\textsf{T} Q }[/math]. Multiplying by [math]\displaystyle{ Q }[/math] and letting [math]\displaystyle{ x = Q^\textsf{T} y }[/math], we get [math]\displaystyle{ Q(M - \lambda N)Q^\textsf{T} y = 0 }[/math], which can be rewritten as [math]\displaystyle{ \left(QMQ^\textsf{T}\right)y = \lambda y }[/math] where [math]\displaystyle{ y^\textsf{T} y = 1 }[/math]. Manipulation now yields [math]\displaystyle{ MX = NX\Lambda }[/math] where [math]\displaystyle{ X }[/math] is a matrix having as columns the generalized eigenvectors and [math]\displaystyle{ \Lambda }[/math] is a diagonal matrix of the generalized eigenvalues. Now premultiplication with [math]\displaystyle{ X^\textsf{T} }[/math] gives the final result: [math]\displaystyle{ X^\textsf{T} MX = \Lambda }[/math] and [math]\displaystyle{ X^\textsf{T} NX = I }[/math], but note that this is no longer an orthogonal diagonalization with respect to the inner product where [math]\displaystyle{ y^\textsf{T} y = 1 }[/math]. In fact, we diagonalized [math]\displaystyle{ M }[/math] with respect to the inner product induced by [math]\displaystyle{ N }[/math].

Note that this result does not contradict what is said on simultaneous diagonalization in the article Diagonalizable matrix, which refers to simultaneous diagonalization by a similarity transformation. Our result here is more akin to a simultaneous diagonalization of two quadratic forms, and is useful for optimization of one form under conditions on the other.

Properties

Induced partial ordering

For arbitrary square matrices [math]\displaystyle{ M }[/math], [math]\displaystyle{ N }[/math] we write [math]\displaystyle{ M \ge N }[/math] if [math]\displaystyle{ M - N \ge 0 }[/math] i.e., [math]\displaystyle{ M - N }[/math] is positive semi-definite. This defines a partial ordering on the set of all square matrices. One can similarly define a strict partial ordering [math]\displaystyle{ M \gt N }[/math]. The ordering is called the Loewner order.

Inverse of positive definite matrix

Every positive definite matrix is invertible and its inverse is also positive definite. If [math]\displaystyle{ M \geq N \gt 0 }[/math] then [math]\displaystyle{ N^{-1} \geq M^{-1} \gt 0 }[/math]. Moreover, by the min-max theorem, the kth largest eigenvalue of [math]\displaystyle{ M }[/math] is greater than the kth largest eigenvalue of [math]\displaystyle{ N }[/math].

Scaling

If [math]\displaystyle{ M }[/math] is positive definite and [math]\displaystyle{ r \gt 0 }[/math] is a real number, then [math]\displaystyle{ rM }[/math] is positive definite.

Addition

If [math]\displaystyle{ M }[/math] and [math]\displaystyle{ N }[/math] are positive definite, then the sum [math]\displaystyle{ M + N }[/math] is also positive definite.

Multiplication

• If [math]\displaystyle{ M }[/math] and [math]\displaystyle{ N }[/math] are positive definite, then the products [math]\displaystyle{ MNM }[/math] and [math]\displaystyle{ NMN }[/math] are also positive definite. If [math]\displaystyle{ MN = NM }[/math], then [math]\displaystyle{ MN }[/math] is also positive definite.

• If [math]\displaystyle{ M }[/math] is positive semidefinite, then [math]\displaystyle{ A^* MA }[/math] is positive semidefinite for any (possibly rectangular) matrix [math]\displaystyle{ A }[/math]. If [math]\displaystyle{ M }[/math] is positive definite and [math]\displaystyle{ Q }[/math] has full column rank, then [math]\displaystyle{ A^* M A }[/math] is positive definite.

Submatrices

Every principal submatrix of a positive definite matrix is positive definite.

Trace

The diagonal entries [math]\displaystyle{ m_{ii} }[/math] of a positive definite matrix are real and non-negative. As a consequence the trace, [math]\displaystyle{ \operatorname{tr}(M) \ge 0 }[/math]. Furthermore, since every principal sub-matrix (in particular, 2-by-2) is positive definite,

[math]\displaystyle{ \left|m_{ij}\right| \leq \sqrt{m_{ii}m_{jj}} \leq \frac{m_{ii} + m_{jj}}{2} \quad \forall i, j }[/math]

and thus

[math]\displaystyle{ \max_{i,j} \left|m_{ij}\right| \leq \max_i\left|m_{ii}\right| }[/math]

An [math]\displaystyle{ n \times n }[/math] Hermitian matrix [math]\displaystyle{ M }[/math] is positive definite if it satisfies the following trace inequalities:

[math]\displaystyle{ \operatorname{tr}(M) \gt 0 \quad \mathrm{and} \quad \frac{(\operatorname{tr}(M))^2}{\operatorname{tr}(M^2)} \gt n-1. }[/math]

Hadamard product

If [math]\displaystyle{ M, N \geq 0 }[/math], although [math]\displaystyle{ MN }[/math] is not necessary positive semidefinite, the Hadamard product is, [math]\displaystyle{ M \circ N \geq 0 }[/math] (this result is often called the Schur product theorem).

Regarding the Hadamard product of two positive semidefinite matrices [math]\displaystyle{ M = (m_{ij}) \geq 0 }[/math], [math]\displaystyle{ N \geq 0 }[/math], there are two notable inequalities:

• Oppenheim's inequality: [math]\displaystyle{ \det(M \circ N) \geq \det (N) \prod\nolimits_i m_{ii}. }[/math]
• [math]\displaystyle{ \det(M \circ N) \geq \det(M) \det(N) }[/math].

Kronecker product

If [math]\displaystyle{ M, N \geq 0 }[/math], although [math]\displaystyle{ MN }[/math] is not necessary positive semidefinite, the Kronecker product [math]\displaystyle{ M \otimes N \geq 0 }[/math].

Frobenius product

If [math]\displaystyle{ M, N \geq 0 }[/math], although [math]\displaystyle{ MN }[/math] is not necessary positive semidefinite, the Frobenius product [math]\displaystyle{ M : N \geq 0 }[/math] (Lancaster–Tismenetsky, The Theory of Matrices, p. 218).

Convexity

The set of positive semidefinite symmetric matrices is convex. That is, if [math]\displaystyle{ M }[/math] and [math]\displaystyle{ N }[/math] are positive semidefinite, then for any [math]\displaystyle{ \alpha }[/math] between 0 and 1, [math]\displaystyle{ \alpha M + (1 - \alpha) N }[/math] is also positive semidefinite. For any vector [math]\displaystyle{ x }[/math]:

[math]\displaystyle{ x^\textsf{T} \left(\alpha M + (1 - \alpha)N\right)x = \alpha x^\textsf{T} Mx + (1 - \alpha) x^\textsf{T} Nx \geq 0. }[/math]

This property guarantees that semidefinite programming problems converge to a globally optimal solution.

Relation with cosine

The positive-definiteness of a matrix [math]\displaystyle{ A }[/math] expresses that the angle [math]\displaystyle{ \theta }[/math] between any vector [math]\displaystyle{ x }[/math] and its image [math]\displaystyle{ Ax }[/math] is always [math]\displaystyle{ -\pi / 2 \lt \theta \lt +\pi / 2 }[/math]:

[math]\displaystyle{ \cos\theta = \frac{x^{T}Ax}{\left\Vert x \right\Vert \left\Vert Ax \right\Vert}=\frac{\langle x,Ax \rangle}{\left\Vert x \right\Vert \left\Vert Ax \right\Vert} , \theta=\theta(x,Ax)=\widehat{x,Ax}= \text{the angle between } x \text{ and } Ax }[/math]

Further properties

1. If [math]\displaystyle{ M }[/math] is a symmetric Toeplitz matrix, i.e. the entries [math]\displaystyle{ m_{ij} }[/math] are given as a function of their absolute index differences: [math]\displaystyle{ m_{ij} = h(|i-j|) }[/math], and the strict inequality

   [math]\displaystyle{ \sum\nolimits_{j \neq 0} \left|h(j)\right| \lt h(0) }[/math]

   holds, then [math]\displaystyle{ M }[/math] is strictly positive definite.
2. Let [math]\displaystyle{ M \gt 0 }[/math] and [math]\displaystyle{ N }[/math] Hermitian. If [math]\displaystyle{ MN + NM \ge 0 }[/math] (resp., [math]\displaystyle{ MN + NM \gt 0 }[/math]) then [math]\displaystyle{ N \ge 0 }[/math] (resp., [math]\displaystyle{ N \gt 0 }[/math]).
3. If [math]\displaystyle{ M \gt 0 }[/math] is real, then there is a [math]\displaystyle{ \delta \gt 0 }[/math] such that [math]\displaystyle{ M\gt \delta I }[/math], where [math]\displaystyle{ I }[/math] is the identity matrix.
4. If [math]\displaystyle{ M_k }[/math] denotes the leading [math]\displaystyle{ k \times k }[/math] minor, [math]\displaystyle{ \det\left(M_k\right)/\det\left(M_{k-1}\right) }[/math] is the kth pivot during LU decomposition.
5. A matrix is negative definite if its k-th order leading principal minor is negative when [math]\displaystyle{ k }[/math] is odd, and positive when [math]\displaystyle{ k }[/math] is even.

A Hermitian matrix is positive semidefinite if and only if all of its principal minors are nonnegative. It is however not enough to consider the leading principal minors only, as is checked on the diagonal matrix with entries 0 and −1.

Block matrices

A positive [math]\displaystyle{ 2n \times 2n }[/math] matrix may also be defined by blocks:

[math]\displaystyle{ M = \begin{bmatrix} A & B \\ C & D \end{bmatrix} }[/math]

where each block is [math]\displaystyle{ n \times n }[/math]. By applying the positivity condition, it immediately follows that [math]\displaystyle{ A }[/math] and [math]\displaystyle{ D }[/math] are hermitian, and [math]\displaystyle{ C = B^* }[/math].

We have that [math]\displaystyle{ z^* Mz \ge 0 }[/math] for all complex [math]\displaystyle{ z }[/math], and in particular for [math]\displaystyle{ z = [v, 0]^\textsf{T} }[/math]. Then

[math]\displaystyle{ \begin{bmatrix} v^* & 0 \end{bmatrix} \begin{bmatrix} A & B \\ B^* & D \end{bmatrix} \begin{bmatrix} v \\ 0 \end{bmatrix} = v^* Av \ge 0. }[/math]

A similar argument can be applied to [math]\displaystyle{ D }[/math], and thus we conclude that both [math]\displaystyle{ A }[/math] and [math]\displaystyle{ D }[/math] must be positive definite matrices, as well.

Converse results can be proved with stronger conditions on the blocks, for instance using the Schur complement.

Local extrema

A general quadratic form [math]\displaystyle{ f(\mathbf{x}) }[/math] on [math]\displaystyle{ n }[/math] real variables [math]\displaystyle{ x_1, \ldots, x_n }[/math] can always be written as [math]\displaystyle{ \mathbf{x}^\textsf{T} M \mathbf{x} }[/math] where [math]\displaystyle{ \mathbf{x} }[/math] is the column vector with those variables, and [math]\displaystyle{ M }[/math] is a symmetric real matrix. Therefore, the matrix being positive definite means that [math]\displaystyle{ f }[/math] has a unique minimum (zero) when [math]\displaystyle{ \mathbf{x} }[/math] is zero, and is strictly positive for any other [math]\displaystyle{ \mathbf{x} }[/math].

More generally, a twice-differentiable real function [math]\displaystyle{ f }[/math] on [math]\displaystyle{ n }[/math] real variables has local minimum at arguments [math]\displaystyle{ x_1, \ldots, x_n }[/math] if its gradient is zero and its Hessian (the matrix of all second derivatives) is positive semi-definite at that point. Similar statements can be made for negative definite and semi-definite matrices.

Covariance

In statistics, the covariance matrix of a multivariate probability distribution is always positive semi-definite; and it is positive definite unless one variable is an exact linear function of the others. Conversely, every positive semi-definite matrix is the covariance matrix of some multivariate distribution.

Extension for non-Hermitian square matrices

The definition of positive definite can be generalized by designating any complex matrix [math]\displaystyle{ M }[/math] (e.g. real non-symmetric) as positive definite if [math]\displaystyle{ \Re\left(z^* Mz\right) \gt 0 }[/math] for all non-zero complex vectors [math]\displaystyle{ z }[/math], where [math]\displaystyle{ \Re(c) }[/math] denotes the real part of a complex number [math]\displaystyle{ c }[/math]. Only the Hermitian part [math]\displaystyle{ \tfrac{1}{2}\left(M + M^*\right) }[/math] determines whether the matrix is positive definite, and is assessed in the narrower sense above. Similarly, If [math]\displaystyle{ x }[/math] and [math]\displaystyle{ M }[/math] are real, we have [math]\displaystyle{ x^\textsf{T} M x \gt 0 }[/math] for all real nonzero vectors [math]\displaystyle{ x }[/math] if and only if the symmetric part [math]\displaystyle{ \tfrac{1}{2}\left(M + M^\textsf{T}\right) }[/math] is positive definite in the narrower sense. It is immediately clear that [math]\displaystyle{ x^\textsf{T} M x=x_iM_{ij}x_j }[/math]is insensitive to transposition of M.

Consequently, a non-symmetric real matrix with only positive eigenvalues does not need to be positive definite. For example, the matrix [math]\displaystyle{ M = \left[\begin{smallmatrix} 4 & 9 \\ 1 & 4 \end{smallmatrix}\right] }[/math] has positive eigenvalues yet is not positive definite; in particular a negative value of [math]\displaystyle{ x^\textsf{T} Mx }[/math] is obtained with the choice [math]\displaystyle{ x = \left[\begin{smallmatrix} -1 \\ 1 \end{smallmatrix}\right] }[/math] (which is the eigenvector associated with the negative eigenvalue of the symmetric part of [math]\displaystyle{ M }[/math]).

In summary, the distinguishing feature between the real and complex case is that, a bounded positive operator on a complex Hilbert space is necessarily Hermitian, or self adjoint. The general claim can be argued using the polarization identity. That is no longer true in the real case.

Applications

Heat conductivity matrix

Fourier's law of heat conduction, giving heat flux [math]\displaystyle{ q }[/math] in terms of the temperature gradient [math]\displaystyle{ g = \nabla T }[/math] is written for anisotropic media as [math]\displaystyle{ q = -Kg }[/math], in which [math]\displaystyle{ K }[/math] is the symmetric thermal conductivity matrix. The negative is inserted in Fourier's law to reflect the expectation that heat will always flow from hot to cold. In other words, since the temperature gradient [math]\displaystyle{ g }[/math] always points from cold to hot, the heat flux [math]\displaystyle{ q }[/math] is expected to have a negative inner product with [math]\displaystyle{ g }[/math] so that [math]\displaystyle{ q^\textsf{T}g \lt 0 }[/math]. Substituting Fourier's law then gives this expectation as [math]\displaystyle{ g^\textsf{T}Kg \gt 0 }[/math], implying that the conductivity matrix should be positive definite.

See also

• Covariance matrix
• M-matrix
• Positive-definite function
• Positive-definite kernel
• Schur complement
• Sylvester's criterion
• Numerical range

References

• Horn, Roger A.; Johnson, Charles R. (2013). Matrix Analysis (2nd ed.). Cambridge University Press. ISBN 978-0-521-54823-6. 
• Bhatia, Rajendra (2007). Positive definite matrices. Princeton Series in Applied Mathematics. ISBN 978-0-691-12918-1. 
• Bernstein, B.; Toupin, R. A. (1962). "Some Properties of the Hessian Matrix of a Strictly Convex Function". Journal für die reine und angewandte Mathematik 210: 67–72. doi:10.1515/crll.1962.210.65. 

External links

• Hazewinkel, Michiel, ed. (2001), "Positive-definite form", Encyclopedia of Mathematics, Springer Science+Business Media B.V. / Kluwer Academic Publishers, ISBN 978-1-55608-010-4, https://www.encyclopediaofmath.org/index.php?title=p/p073880 
• Wolfram MathWorld: Positive Definite Matrix

de:Definitheit#Definitheit von Matrizen