Generalized eigenvector

Short description: Vector satisfying some of the criteria of an eigenvector

In linear algebra, a generalized eigenvector of an [math]\displaystyle{ n\times n }[/math] matrix [math]\displaystyle{ A }[/math] is a vector which satisfies certain criteria which are more relaxed than those for an (ordinary) eigenvector.

Let [math]\displaystyle{ V }[/math] be an [math]\displaystyle{ n }[/math]-dimensional vector space and let [math]\displaystyle{ A }[/math] be the matrix representation of a linear map from [math]\displaystyle{ V }[/math] to [math]\displaystyle{ V }[/math] with respect to some ordered basis.

There may not always exist a full set of [math]\displaystyle{ n }[/math] linearly independent eigenvectors of [math]\displaystyle{ A }[/math] that form a complete basis for [math]\displaystyle{ V }[/math]. That is, the matrix [math]\displaystyle{ A }[/math] may not be diagonalizable. This happens when the algebraic multiplicity of at least one eigenvalue [math]\displaystyle{ \lambda_i }[/math] is greater than its geometric multiplicity (the nullity of the matrix [math]\displaystyle{ (A-\lambda_i I) }[/math], or the dimension of its nullspace). In this case, [math]\displaystyle{ \lambda_i }[/math] is called a defective eigenvalue and [math]\displaystyle{ A }[/math] is called a defective matrix.

A generalized eigenvector [math]\displaystyle{ x_i }[/math] corresponding to [math]\displaystyle{ \lambda_i }[/math], together with the matrix [math]\displaystyle{ (A-\lambda_i I) }[/math] generate a Jordan chain of linearly independent generalized eigenvectors which form a basis for an invariant subspace of [math]\displaystyle{ V }[/math].

Using generalized eigenvectors, a set of linearly independent eigenvectors of [math]\displaystyle{ A }[/math] can be extended, if necessary, to a complete basis for [math]\displaystyle{ V }[/math]. This basis can be used to determine an "almost diagonal matrix" [math]\displaystyle{ J }[/math] in Jordan normal form, similar to [math]\displaystyle{ A }[/math], which is useful in computing certain matrix functions of [math]\displaystyle{ A }[/math]. The matrix [math]\displaystyle{ J }[/math] is also useful in solving the system of linear differential equations [math]\displaystyle{ \mathbf x' = A \mathbf x, }[/math] where [math]\displaystyle{ A }[/math] need not be diagonalizable.

The dimension of the generalized eigenspace corresponding to a given eigenvalue [math]\displaystyle{ \lambda }[/math] is the algebraic multiplicity of [math]\displaystyle{ \lambda }[/math].

Overview and definition

There are several equivalent ways to define an ordinary eigenvector. For our purposes, an eigenvector [math]\displaystyle{ \mathbf u }[/math] associated with an eigenvalue [math]\displaystyle{ \lambda }[/math] of an [math]\displaystyle{ n }[/math] × [math]\displaystyle{ n }[/math] matrix [math]\displaystyle{ A }[/math] is a nonzero vector for which [math]\displaystyle{ (A - \lambda I) \mathbf u = \mathbf 0 }[/math], where [math]\displaystyle{ I }[/math] is the [math]\displaystyle{ n }[/math] × [math]\displaystyle{ n }[/math] identity matrix and [math]\displaystyle{ \mathbf 0 }[/math] is the zero vector of length [math]\displaystyle{ n }[/math]. That is, [math]\displaystyle{ \mathbf u }[/math] is in the kernel of the transformation [math]\displaystyle{ (A - \lambda I) }[/math]. If [math]\displaystyle{ A }[/math] has [math]\displaystyle{ n }[/math] linearly independent eigenvectors, then [math]\displaystyle{ A }[/math] is similar to a diagonal matrix [math]\displaystyle{ D }[/math]. That is, there exists an invertible matrix [math]\displaystyle{ M }[/math] such that [math]\displaystyle{ A }[/math] is diagonalizable through the similarity transformation [math]\displaystyle{ D = M^{-1}AM }[/math]. The matrix [math]\displaystyle{ D }[/math] is called a spectral matrix for [math]\displaystyle{ A }[/math]. The matrix [math]\displaystyle{ M }[/math] is called a modal matrix for [math]\displaystyle{ A }[/math]. Diagonalizable matrices are of particular interest since matrix functions of them can be computed easily.

On the other hand, if [math]\displaystyle{ A }[/math] does not have [math]\displaystyle{ n }[/math] linearly independent eigenvectors associated with it, then [math]\displaystyle{ A }[/math] is not diagonalizable.

Definition: A vector [math]\displaystyle{ \mathbf x_m }[/math] is a generalized eigenvector of rank m of the matrix [math]\displaystyle{ A }[/math] and corresponding to the eigenvalue [math]\displaystyle{ \lambda }[/math] if

[math]\displaystyle{ (A - \lambda I)^m \mathbf x_m = \mathbf 0 }[/math]

but

[math]\displaystyle{ (A - \lambda I)^{m-1} \mathbf x_m \ne \mathbf 0. }[/math]

Clearly, a generalized eigenvector of rank 1 is an ordinary eigenvector. Every [math]\displaystyle{ n }[/math] × [math]\displaystyle{ n }[/math] matrix [math]\displaystyle{ A }[/math] has [math]\displaystyle{ n }[/math] linearly independent generalized eigenvectors associated with it and can be shown to be similar to an "almost diagonal" matrix [math]\displaystyle{ J }[/math] in Jordan normal form. That is, there exists an invertible matrix [math]\displaystyle{ M }[/math] such that [math]\displaystyle{ J = M^{-1}AM }[/math]. The matrix [math]\displaystyle{ M }[/math] in this case is called a generalized modal matrix for [math]\displaystyle{ A }[/math]. If [math]\displaystyle{ \lambda }[/math] is an eigenvalue of algebraic multiplicity [math]\displaystyle{ \mu }[/math], then [math]\displaystyle{ A }[/math] will have [math]\displaystyle{ \mu }[/math] linearly independent generalized eigenvectors corresponding to [math]\displaystyle{ \lambda }[/math]. These results, in turn, provide a straightforward method for computing certain matrix functions of [math]\displaystyle{ A }[/math].

Note: For an [math]\displaystyle{ n \times n }[/math] matrix [math]\displaystyle{ A }[/math] over a field [math]\displaystyle{ F }[/math] to be expressed in Jordan normal form, all eigenvalues of [math]\displaystyle{ A }[/math] must be in [math]\displaystyle{ F }[/math]. That is, the characteristic polynomial [math]\displaystyle{ f(x) }[/math] must factor completely into linear factors. For example, if [math]\displaystyle{ A }[/math] has real-valued elements, then it may be necessary for the eigenvalues and the components of the eigenvectors to have complex values.

The set spanned by all generalized eigenvectors for a given [math]\displaystyle{ \lambda }[/math] forms the generalized eigenspace for [math]\displaystyle{ \lambda }[/math].

Examples

Here are some examples to illustrate the concept of generalized eigenvectors. Some of the details will be described later.

Example 1

This example is simple but clearly illustrates the point. This type of matrix is used frequently in textbooks. Suppose

[math]\displaystyle{ A = \begin{pmatrix} 1 & 1\\ 0 & 1 \end{pmatrix}. }[/math]

Then there is only one eigenvalue, [math]\displaystyle{ \lambda = 1 }[/math], and its algebraic multiplicity is [math]\displaystyle{ m=2 }[/math].

Notice that this matrix is in Jordan normal form but is not diagonal. Hence, this matrix is not diagonalizable. Since there is one superdiagonal entry, there will be one generalized eigenvector of rank greater than 1 (or one could note that the vector space [math]\displaystyle{ V }[/math] is of dimension 2, so there can be at most one generalized eigenvector of rank greater than 1). Alternatively, one could compute the dimension of the nullspace of [math]\displaystyle{ A - \lambda I }[/math] to be [math]\displaystyle{ p=1 }[/math], and thus there are [math]\displaystyle{ m-p=1 }[/math] generalized eigenvectors of rank greater than 1.

The ordinary eigenvector [math]\displaystyle{ \mathbf v_1=\begin{pmatrix}1 \\0 \end{pmatrix} }[/math] is computed as usual (see the eigenvector page for examples). Using this eigenvector, we compute the generalized eigenvector [math]\displaystyle{ \mathbf v_2 }[/math] by solving

[math]\displaystyle{ (A-\lambda I) \mathbf v_2 = \mathbf v_1. }[/math]

Writing out the values:

[math]\displaystyle{ \left(\begin{pmatrix} 1 & 1\\ 0 & 1 \end{pmatrix} - 1 \begin{pmatrix} 1 & 0\\ 0 & 1 \end{pmatrix}\right)\begin{pmatrix}v_{21} \\v_{22} \end{pmatrix} = \begin{pmatrix} 0 & 1\\ 0 & 0 \end{pmatrix} \begin{pmatrix}v_{21} \\v_{22} \end{pmatrix} = \begin{pmatrix}1 \\0 \end{pmatrix}. }[/math]

This simplifies to

[math]\displaystyle{ v_{22}= 1. }[/math]

The element [math]\displaystyle{ v_{21} }[/math] has no restrictions. The generalized eigenvector of rank 2 is then [math]\displaystyle{ \mathbf v_2=\begin{pmatrix}a \\1 \end{pmatrix} }[/math], where a can have any scalar value. The choice of a = 0 is usually the simplest.

Note that

[math]\displaystyle{ (A-\lambda I) \mathbf v_2 = \begin{pmatrix} 0 & 1\\ 0 & 0 \end{pmatrix} \begin{pmatrix}a \\1 \end{pmatrix} = \begin{pmatrix}1 \\0 \end{pmatrix} = \mathbf v_1, }[/math]

so that [math]\displaystyle{ \mathbf v_2 }[/math] is a generalized eigenvector, because

[math]\displaystyle{ (A-\lambda I)^2 \mathbf v_2 = (A-\lambda I) [(A-\lambda I)\mathbf v_2] =(A-\lambda I) \mathbf v_1 = \begin{pmatrix} 0 & 1\\ 0 & 0 \end{pmatrix} \begin{pmatrix}1 \\0 \end{pmatrix} = \begin{pmatrix}0 \\0 \end{pmatrix} = \mathbf 0, }[/math]

so that [math]\displaystyle{ \mathbf v_1 }[/math] is an ordinary eigenvector, and that [math]\displaystyle{ \mathbf v_1 }[/math] and [math]\displaystyle{ \mathbf v_2 }[/math] are linearly independent and hence constitute a basis for the vector space [math]\displaystyle{ V }[/math].

Example 2

This example is more complex than Example 1. Unfortunately, it is a little difficult to construct an interesting example of low order. The matrix

[math]\displaystyle{ A = \begin{pmatrix} 1 & 0 & 0 & 0 & 0 \\ 3 & 1 & 0 & 0 & 0 \\ 6 & 3 & 2 & 0 & 0 \\ 10 & 6 & 3 & 2 & 0 \\ 15 & 10 & 6 & 3 & 2 \end{pmatrix} }[/math]

has eigenvalues [math]\displaystyle{ \lambda_1 = 1 }[/math] and [math]\displaystyle{ \lambda_2 = 2 }[/math] with algebraic multiplicities [math]\displaystyle{ \mu_1 = 2 }[/math] and [math]\displaystyle{ \mu_2 = 3 }[/math], but geometric multiplicities [math]\displaystyle{ \gamma_1 = 1 }[/math] and [math]\displaystyle{ \gamma_2 = 1 }[/math].

The generalized eigenspaces of [math]\displaystyle{ A }[/math] are calculated below. [math]\displaystyle{ \mathbf x_1 }[/math] is the ordinary eigenvector associated with [math]\displaystyle{ \lambda_1 }[/math]. [math]\displaystyle{ \mathbf x_2 }[/math] is a generalized eigenvector associated with [math]\displaystyle{ \lambda_1 }[/math]. [math]\displaystyle{ \mathbf y_1 }[/math] is the ordinary eigenvector associated with [math]\displaystyle{ \lambda_2 }[/math]. [math]\displaystyle{ \mathbf y_2 }[/math] and [math]\displaystyle{ \mathbf y_3 }[/math] are generalized eigenvectors associated with [math]\displaystyle{ \lambda_2 }[/math].

[math]\displaystyle{ (A-1 I) \mathbf x_1 = \begin{pmatrix} 0 & 0 & 0 & 0 & 0 \\ 3 & 0 & 0 & 0 & 0 \\ 6 & 3 & 1 & 0 & 0 \\ 10 & 6 & 3 & 1 & 0 \\ 15 & 10 & 6 & 3 & 1 \end{pmatrix}\begin{pmatrix} 0 \\ 3 \\ -9 \\ 9 \\ -3 \end{pmatrix} = \begin{pmatrix} 0 \\ 0 \\ 0 \\ 0 \\ 0 \end{pmatrix} = \mathbf 0 , }[/math]

[math]\displaystyle{ (A - 1 I) \mathbf x_2 = \begin{pmatrix} 0 & 0 & 0 & 0 & 0 \\ 3 & 0 & 0 & 0 & 0 \\ 6 & 3 & 1 & 0 & 0 \\ 10 & 6 & 3 & 1 & 0 \\ 15 & 10 & 6 & 3 & 1 \end{pmatrix} \begin{pmatrix} 1 \\ -15 \\ 30 \\ -1 \\ -45 \end{pmatrix} = \begin{pmatrix} 0 \\ 3 \\ -9 \\ 9 \\ -3 \end{pmatrix} = \mathbf x_1 , }[/math]

[math]\displaystyle{ (A - 2 I) \mathbf y_1 = \begin{pmatrix} -1 & 0 & 0 & 0 & 0 \\ 3 & -1 & 0 & 0 & 0 \\ 6 & 3 & 0 & 0 & 0 \\ 10 & 6 & 3 & 0 & 0 \\ 15 & 10 & 6 & 3 & 0 \end{pmatrix} \begin{pmatrix} 0 \\ 0 \\ 0 \\ 0 \\ 9 \end{pmatrix} = \begin{pmatrix} 0 \\ 0 \\ 0 \\ 0 \\ 0 \end{pmatrix} = \mathbf 0 , }[/math]

[math]\displaystyle{ (A - 2 I) \mathbf y_2 = \begin{pmatrix} -1 & 0 & 0 & 0 & 0 \\ 3 & -1 & 0 & 0 & 0 \\ 6 & 3 & 0 & 0 & 0 \\ 10 & 6 & 3 & 0 & 0 \\ 15 & 10 & 6 & 3 & 0 \end{pmatrix} \begin{pmatrix} 0 \\ 0 \\ 0 \\ 3 \\ 0 \end{pmatrix} = \begin{pmatrix} 0 \\ 0 \\ 0 \\ 0 \\ 9 \end{pmatrix} = \mathbf y_1 , }[/math]

[math]\displaystyle{ (A - 2 I) \mathbf y_3 = \begin{pmatrix} -1 & 0 & 0 & 0 & 0 \\ 3 & -1 & 0 & 0 & 0 \\ 6 & 3 & 0 & 0 & 0 \\ 10 & 6 & 3 & 0 & 0 \\ 15 & 10 & 6 & 3 & 0 \end{pmatrix} \begin{pmatrix} 0 \\ 0 \\ 1 \\ -2 \\ 0 \end{pmatrix} = \begin{pmatrix} 0 \\ 0 \\ 0 \\ 3 \\ 0 \end{pmatrix} = \mathbf y_2 . }[/math]

This results in a basis for each of the generalized eigenspaces of [math]\displaystyle{ A }[/math]. Together the two chains of generalized eigenvectors span the space of all 5-dimensional column vectors.

[math]\displaystyle{ \left\{ \mathbf x_1, \mathbf x_2 \right\} = \left\{ \begin{pmatrix} 0 \\ 3 \\ -9 \\ 9 \\ -3 \end{pmatrix} \begin{pmatrix} 1 \\ -15 \\ 30 \\ -1 \\ -45 \end{pmatrix} \right\}, \left\{ \mathbf y_1, \mathbf y_2, \mathbf y_3 \right\} = \left\{ \begin{pmatrix} 0 \\ 0 \\ 0 \\ 0 \\ 9 \end{pmatrix} \begin{pmatrix} 0 \\ 0 \\ 0 \\ 3 \\ 0 \end{pmatrix} \begin{pmatrix} 0 \\ 0 \\ 1 \\ -2 \\ 0 \end{pmatrix} \right\}. }[/math]

An "almost diagonal" matrix [math]\displaystyle{ J }[/math] in Jordan normal form, similar to [math]\displaystyle{ A }[/math] is obtained as follows:

[math]\displaystyle{ M = \begin{pmatrix} \mathbf x_1 & \mathbf x_2 & \mathbf y_1 & \mathbf y_2 & \mathbf y_3 \end{pmatrix} = \begin{pmatrix} 0 & 1 & 0 &0& 0 \\ 3 & -15 & 0 &0& 0 \\ -9 & 30 & 0 &0& 1 \\ 9 & -1 & 0 &3& -2 \\ -3 & -45 & 9 &0& 0 \end{pmatrix}, }[/math]

[math]\displaystyle{ J = \begin{pmatrix} 1 & 1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 & 0 \\ 0 & 0 & 2 & 1 & 0 \\ 0 & 0 & 0 & 2 & 1 \\ 0 & 0 & 0 & 0 & 2 \end{pmatrix}, }[/math]

where [math]\displaystyle{ M }[/math] is a generalized modal matrix for [math]\displaystyle{ A }[/math], the columns of [math]\displaystyle{ M }[/math] are a canonical basis for [math]\displaystyle{ A }[/math], and [math]\displaystyle{ AM = MJ }[/math].

Jordan chains

Definition: Let [math]\displaystyle{ \mathbf x_m }[/math] be a generalized eigenvector of rank m corresponding to the matrix [math]\displaystyle{ A }[/math] and the eigenvalue [math]\displaystyle{ \lambda }[/math]. The chain generated by [math]\displaystyle{ \mathbf x_m }[/math] is a set of vectors [math]\displaystyle{ \left\{ \mathbf x_m, \mathbf x_{m-1}, \dots , \mathbf x_1 \right\} }[/math] given by

[math]\displaystyle{ \mathbf x_{m-1} = (A - \lambda I) \mathbf x_m, }[/math] [math]\displaystyle{ \mathbf x_{m-2} = (A - \lambda I)^2 \mathbf x_m = (A - \lambda I) \mathbf x_{m-1}, }[/math] [math]\displaystyle{ \mathbf x_{m-3} = (A - \lambda I)^3 \mathbf x_m = (A - \lambda I) \mathbf x_{m-2}, }[/math] [math]\displaystyle{ \vdots }[/math] [math]\displaystyle{ \mathbf x_1 = (A - \lambda I)^{m-1} \mathbf x_m = (A - \lambda I) \mathbf x_2. }[/math]

(1)

Thus, in general,

[math]\displaystyle{ \mathbf x_j = (A - \lambda I)^{m-j} \mathbf x_m = (A - \lambda I) \mathbf x_{j+1} \qquad (j = 1, 2, \dots , m - 1). }[/math]

(2)

The vector [math]\displaystyle{ \mathbf x_j }[/math], given by (2), is a generalized eigenvector of rank j corresponding to the eigenvalue [math]\displaystyle{ \lambda }[/math]. A chain is a linearly independent set of vectors.

Canonical basis

Definition: A set of n linearly independent generalized eigenvectors is a canonical basis if it is composed entirely of Jordan chains.

Thus, once we have determined that a generalized eigenvector of rank m is in a canonical basis, it follows that the m − 1 vectors [math]\displaystyle{ \mathbf x_{m-1}, \mathbf x_{m-2}, \ldots , \mathbf x_1 }[/math] that are in the Jordan chain generated by [math]\displaystyle{ \mathbf x_m }[/math] are also in the canonical basis.

Let [math]\displaystyle{ \lambda_i }[/math] be an eigenvalue of [math]\displaystyle{ A }[/math] of algebraic multiplicity [math]\displaystyle{ \mu_i }[/math]. First, find the ranks (matrix ranks) of the matrices [math]\displaystyle{ (A - \lambda_i I), (A - \lambda_i I)^2, \ldots , (A - \lambda_i I)^{m_i} }[/math]. The integer [math]\displaystyle{ m_i }[/math] is determined to be the first integer for which [math]\displaystyle{ (A - \lambda_i I)^{m_i} }[/math] has rank [math]\displaystyle{ n - \mu_i }[/math] (n being the number of rows or columns of [math]\displaystyle{ A }[/math], that is, [math]\displaystyle{ A }[/math] is n × n).

Now define

[math]\displaystyle{ \rho_k = \operatorname{rank}(A - \lambda_i I)^{k-1} - \operatorname{rank}(A - \lambda_i I)^k \qquad (k = 1, 2, \ldots , m_i). }[/math]

The variable [math]\displaystyle{ \rho_k }[/math] designates the number of linearly independent generalized eigenvectors of rank k corresponding to the eigenvalue [math]\displaystyle{ \lambda_i }[/math] that will appear in a canonical basis for [math]\displaystyle{ A }[/math]. Note that

[math]\displaystyle{ \operatorname{rank}(A - \lambda_i I)^0 = \operatorname{rank}(I) = n }[/math].

Computation of generalized eigenvectors

In the preceding sections we have seen techniques for obtaining the [math]\displaystyle{ n }[/math] linearly independent generalized eigenvectors of a canonical basis for the vector space [math]\displaystyle{ V }[/math] associated with an [math]\displaystyle{ n \times n }[/math] matrix [math]\displaystyle{ A }[/math]. These techniques can be combined into a procedure:

Solve the characteristic equation of [math]\displaystyle{ A }[/math] for eigenvalues [math]\displaystyle{ \lambda_i }[/math] and their algebraic multiplicities [math]\displaystyle{ \mu_i }[/math];

For each [math]\displaystyle{ \lambda_i : }[/math]

Determine [math]\displaystyle{ n - \mu_i }[/math];

Determine [math]\displaystyle{ m_i }[/math];

Determine [math]\displaystyle{ \rho_k }[/math] for [math]\displaystyle{ (k = 1, \ldots , m_i) }[/math];

Determine each Jordan chain for [math]\displaystyle{ \lambda_i }[/math];

Example 3

The matrix

[math]\displaystyle{ A = \begin{pmatrix} 5 & 1 & -2 & 4 \\ 0 & 5 & 2 & 2 \\ 0 & 0 & 5 & 3 \\ 0 & 0 & 0 & 4 \end{pmatrix} }[/math]

has an eigenvalue [math]\displaystyle{ \lambda_1 = 5 }[/math] of algebraic multiplicity [math]\displaystyle{ \mu_1 = 3 }[/math] and an eigenvalue [math]\displaystyle{ \lambda_2 = 4 }[/math] of algebraic multiplicity [math]\displaystyle{ \mu_2 = 1 }[/math]. We also have [math]\displaystyle{ n=4 }[/math]. For [math]\displaystyle{ \lambda_1 }[/math] we have [math]\displaystyle{ n - \mu_1 = 4 - 3 = 1 }[/math].

[math]\displaystyle{ (A - 5I) = \begin{pmatrix} 0 & 1 & -2 & 4 \\ 0 & 0 & 2 & 2 \\ 0 & 0 & 0 & 3 \\ 0 & 0 & 0 & -1 \end{pmatrix}, \qquad \operatorname{rank}(A - 5I) = 3. }[/math]

[math]\displaystyle{ (A - 5I)^2 = \begin{pmatrix} 0 & 0 & 2 & -8 \\ 0 & 0 & 0 & 4 \\ 0 & 0 & 0 & -3 \\ 0 & 0 & 0 & 1 \end{pmatrix}, \qquad \operatorname{rank}(A - 5I)^2 = 2. }[/math]

[math]\displaystyle{ (A - 5I)^3 = \begin{pmatrix} 0 & 0 & 0 & 14 \\ 0 & 0 & 0 & -4 \\ 0 & 0 & 0 & 3 \\ 0 & 0 & 0 & -1 \end{pmatrix}, \qquad \operatorname{rank}(A - 5I)^3 = 1. }[/math]

The first integer [math]\displaystyle{ m_1 }[/math] for which [math]\displaystyle{ (A - 5I)^{m_1} }[/math] has rank [math]\displaystyle{ n - \mu_1 = 1 }[/math] is [math]\displaystyle{ m_1 = 3 }[/math].

We now define

[math]\displaystyle{ \rho_3 = \operatorname{rank}(A - 5I)^2 - \operatorname{rank}(A - 5I)^3 = 2 - 1 = 1 , }[/math]

[math]\displaystyle{ \rho_2 = \operatorname{rank}(A - 5I)^1 - \operatorname{rank}(A - 5I)^2 = 3 - 2 = 1 , }[/math]

[math]\displaystyle{ \rho_1 = \operatorname{rank}(A - 5I)^0 - \operatorname{rank}(A - 5I)^1 = 4 - 3 = 1 . }[/math]

Consequently, there will be three linearly independent generalized eigenvectors; one each of ranks 3, 2 and 1. Since [math]\displaystyle{ \lambda_1 }[/math] corresponds to a single chain of three linearly independent generalized eigenvectors, we know that there is a generalized eigenvector [math]\displaystyle{ \mathbf x_3 }[/math] of rank 3 corresponding to [math]\displaystyle{ \lambda_1 }[/math] such that

[math]\displaystyle{ (A - 5I)^3 \mathbf x_3 = \mathbf 0 }[/math]

(3)

but

[math]\displaystyle{ (A - 5I)^2 \mathbf x_3\neq \mathbf 0 . }[/math]

(4)

Equations (3) and (4) represent linear systems that can be solved for [math]\displaystyle{ \mathbf x_3 }[/math]. Let

[math]\displaystyle{ \mathbf x_3 = \begin{pmatrix} x_{31} \\ x_{32} \\ x_{33} \\ x_{34} \end{pmatrix}. }[/math]

Then

[math]\displaystyle{ (A - 5I)^3 \mathbf x_3 = \begin{pmatrix} 0 & 0 & 0 & 14 \\ 0 & 0 & 0 & -4 \\ 0 & 0 & 0 & 3 \\ 0 & 0 & 0 & -1 \end{pmatrix} \begin{pmatrix} x_{31} \\ x_{32} \\ x_{33} \\ x_{34} \end{pmatrix} = \begin{pmatrix} 14 x_{34} \\ -4 x_{34} \\ 3 x_{34} \\ - x_{34} \end{pmatrix} = \begin{pmatrix} 0 \\ 0 \\ 0 \\ 0 \end{pmatrix} }[/math]

and

[math]\displaystyle{ (A - 5I)^2 \mathbf x_3 = \begin{pmatrix} 0 & 0 & 2 & -8 \\ 0 & 0 & 0 & 4 \\ 0 & 0 & 0 & -3 \\ 0 & 0 & 0 & 1 \end{pmatrix} \begin{pmatrix} x_{31} \\ x_{32} \\ x_{33} \\ x_{34} \end{pmatrix} = \begin{pmatrix} 2 x_{33} - 8 x_{34} \\ 4 x_{34} \\ -3 x_{34} \\ x_{34} \end{pmatrix} \ne \begin{pmatrix} 0 \\ 0 \\ 0 \\ 0 \end{pmatrix}. }[/math]

Thus, in order to satisfy the conditions (3) and (4), we must have [math]\displaystyle{ x_{34} = 0 }[/math] and [math]\displaystyle{ x_{33} \ne 0 }[/math]. No restrictions are placed on [math]\displaystyle{ x_{31} }[/math] and [math]\displaystyle{ x_{32} }[/math]. By choosing [math]\displaystyle{ x_{31} = x_{32} = x_{34} = 0, x_{33} = 1 }[/math], we obtain

[math]\displaystyle{ \mathbf x_3 = \begin{pmatrix} 0 \\ 0 \\ 1 \\ 0 \end{pmatrix} }[/math]

as a generalized eigenvector of rank 3 corresponding to [math]\displaystyle{ \lambda_1 = 5 }[/math]. Note that it is possible to obtain infinitely many other generalized eigenvectors of rank 3 by choosing different values of [math]\displaystyle{ x_{31} }[/math], [math]\displaystyle{ x_{32} }[/math] and [math]\displaystyle{ x_{33} }[/math], with [math]\displaystyle{ x_{33} \ne 0 }[/math]. Our first choice, however, is the simplest.

Now using equations (1), we obtain [math]\displaystyle{ \mathbf x_2 }[/math] and [math]\displaystyle{ \mathbf x_1 }[/math] as generalized eigenvectors of rank 2 and 1, respectively, where

[math]\displaystyle{ \mathbf x_2 = (A - 5I) \mathbf x_3 = \begin{pmatrix} -2 \\ 2 \\ 0 \\ 0 \end{pmatrix}, }[/math]

and

[math]\displaystyle{ \mathbf x_1 = (A - 5I) \mathbf x_2 = \begin{pmatrix} 2 \\ 0 \\ 0 \\ 0 \end{pmatrix}. }[/math]

The simple eigenvalue [math]\displaystyle{ \lambda_2 = 4 }[/math] can be dealt with using standard techniques and has an ordinary eigenvector

[math]\displaystyle{ \mathbf y_1 = \begin{pmatrix} -14 \\ 4 \\ -3 \\ 1 \end{pmatrix}. }[/math]

A canonical basis for [math]\displaystyle{ A }[/math] is

[math]\displaystyle{ \left\{ \mathbf x_3, \mathbf x_2, \mathbf x_1, \mathbf y_1 \right\} = \left\{ \begin{pmatrix} 0 \\ 0 \\ 1 \\ 0 \end{pmatrix} \begin{pmatrix} -2 \\ 2 \\ 0 \\ 0 \end{pmatrix} \begin{pmatrix} 2 \\ 0 \\ 0 \\ 0 \end{pmatrix} \begin{pmatrix} -14 \\ 4 \\ -3 \\ 1 \end{pmatrix} \right\}. }[/math]

[math]\displaystyle{ \mathbf x_1, \mathbf x_2 }[/math] and [math]\displaystyle{ \mathbf x_3 }[/math] are generalized eigenvectors associated with [math]\displaystyle{ \lambda_1 }[/math], while [math]\displaystyle{ \mathbf y_1 }[/math] is the ordinary eigenvector associated with [math]\displaystyle{ \lambda_2 }[/math].

This is a fairly simple example. In general, the numbers [math]\displaystyle{ \rho_k }[/math] of linearly independent generalized eigenvectors of rank [math]\displaystyle{ k }[/math] will not always be equal. That is, there may be several chains of different lengths corresponding to a particular eigenvalue.

Generalized modal matrix

Let [math]\displaystyle{ A }[/math] be an n × n matrix. A generalized modal matrix [math]\displaystyle{ M }[/math] for [math]\displaystyle{ A }[/math] is an n × n matrix whose columns, considered as vectors, form a canonical basis for [math]\displaystyle{ A }[/math] and appear in [math]\displaystyle{ M }[/math] according to the following rules:

• All Jordan chains consisting of one vector (that is, one vector in length) appear in the first columns of [math]\displaystyle{ M }[/math].
• All vectors of one chain appear together in adjacent columns of [math]\displaystyle{ M }[/math].
• Each chain appears in [math]\displaystyle{ M }[/math] in order of increasing rank (that is, the generalized eigenvector of rank 1 appears before the generalized eigenvector of rank 2 of the same chain, which appears before the generalized eigenvector of rank 3 of the same chain, etc.).

Jordan normal form

An example of a matrix in Jordan normal form. The grey blocks are called Jordan blocks.

Let [math]\displaystyle{ V }[/math] be an n-dimensional vector space; let [math]\displaystyle{ \phi }[/math] be a linear map in L(V), the set of all linear maps from [math]\displaystyle{ V }[/math] into itself; and let [math]\displaystyle{ A }[/math] be the matrix representation of [math]\displaystyle{ \phi }[/math] with respect to some ordered basis. It can be shown that if the characteristic polynomial [math]\displaystyle{ f(\lambda) }[/math] of [math]\displaystyle{ A }[/math] factors into linear factors, so that [math]\displaystyle{ f(\lambda) }[/math] has the form

[math]\displaystyle{ f(\lambda) = \pm (\lambda - \lambda_1)^{\mu_1}(\lambda - \lambda_2)^{\mu_2} \cdots (\lambda - \lambda_r)^{\mu_r} , }[/math]

where [math]\displaystyle{ \lambda_1, \lambda_2, \ldots , \lambda_r }[/math] are the distinct eigenvalues of [math]\displaystyle{ A }[/math], then each [math]\displaystyle{ \mu_i }[/math] is the algebraic multiplicity of its corresponding eigenvalue [math]\displaystyle{ \lambda_i }[/math] and [math]\displaystyle{ A }[/math] is similar to a matrix [math]\displaystyle{ J }[/math] in Jordan normal form, where each [math]\displaystyle{ \lambda_i }[/math] appears [math]\displaystyle{ \mu_i }[/math] consecutive times on the diagonal, and the entry directly above each [math]\displaystyle{ \lambda_i }[/math] (that is, on the superdiagonal) is either 0 or 1: the entry above the first occurrence of each [math]\displaystyle{ \lambda_i }[/math] is always 0; all other entries on the superdiagonal are 1. All other entries (that is, off the diagonal and superdiagonal) are 0. (But no ordering is imposed among the eigenvalues, or among the blocks for a given eigenvalue.) The matrix [math]\displaystyle{ J }[/math] is as close as one can come to a diagonalization of [math]\displaystyle{ A }[/math]. If [math]\displaystyle{ A }[/math] is diagonalizable, then all entries above the diagonal are zero. Note that some textbooks have the ones on the subdiagonal, that is, immediately below the main diagonal instead of on the superdiagonal. The eigenvalues are still on the main diagonal.

Every n × n matrix [math]\displaystyle{ A }[/math] is similar to a matrix [math]\displaystyle{ J }[/math] in Jordan normal form, obtained through the similarity transformation [math]\displaystyle{ J = M^{-1}AM }[/math], where [math]\displaystyle{ M }[/math] is a generalized modal matrix for [math]\displaystyle{ A }[/math]. (See Note above.)

Example 4

Find a matrix in Jordan normal form that is similar to

[math]\displaystyle{ A = \begin{pmatrix} 0 & 4 & 2 \\ -3 & 8 & 3 \\ 4 & -8 & -2 \end{pmatrix}. }[/math]

Solution: The characteristic equation of [math]\displaystyle{ A }[/math] is [math]\displaystyle{ (\lambda - 2)^3 = 0 }[/math], hence, [math]\displaystyle{ \lambda = 2 }[/math] is an eigenvalue of algebraic multiplicity three. Following the procedures of the previous sections, we find that

[math]\displaystyle{ \operatorname{rank}(A - 2I) = 1 }[/math]

and

[math]\displaystyle{ \operatorname{rank}(A - 2I)^2 = 0 = n - \mu . }[/math]

Thus, [math]\displaystyle{ \rho_2 = 1 }[/math] and [math]\displaystyle{ \rho_1 = 2 }[/math], which implies that a canonical basis for [math]\displaystyle{ A }[/math] will contain one linearly independent generalized eigenvector of rank 2 and two linearly independent generalized eigenvectors of rank 1, or equivalently, one chain of two vectors [math]\displaystyle{ \left\{ \mathbf x_2, \mathbf x_1 \right\} }[/math] and one chain of one vector [math]\displaystyle{ \left\{ \mathbf y_1 \right\} }[/math]. Designating [math]\displaystyle{ M = \begin{pmatrix} \mathbf y_1 & \mathbf x_1 & \mathbf x_2 \end{pmatrix} }[/math], we find that

[math]\displaystyle{ M = \begin{pmatrix} 2 & 2 & 0 \\ 1 & 3 & 0 \\ 0 & -4 & 1 \end{pmatrix}, }[/math]

and

[math]\displaystyle{ J = \begin{pmatrix} 2 & 0 & 0 \\ 0 & 2 & 1 \\ 0 & 0 & 2 \end{pmatrix}, }[/math]

where [math]\displaystyle{ M }[/math] is a generalized modal matrix for [math]\displaystyle{ A }[/math], the columns of [math]\displaystyle{ M }[/math] are a canonical basis for [math]\displaystyle{ A }[/math], and [math]\displaystyle{ AM = MJ }[/math]. Note that since generalized eigenvectors themselves are not unique, and since some of the columns of both [math]\displaystyle{ M }[/math] and [math]\displaystyle{ J }[/math] may be interchanged, it follows that both [math]\displaystyle{ M }[/math] and [math]\displaystyle{ J }[/math] are not unique.

Example 5

In Example 3, we found a canonical basis of linearly independent generalized eigenvectors for a matrix [math]\displaystyle{ A }[/math]. A generalized modal matrix for [math]\displaystyle{ A }[/math] is

[math]\displaystyle{ M = \begin{pmatrix} \mathbf y_1 & \mathbf x_1 & \mathbf x_2 & \mathbf x_3 \end{pmatrix} = \begin{pmatrix} -14 & 2 & -2 & 0 \\ 4 & 0 & 2 & 0 \\ -3 & 0 & 0 & 1 \\ 1 & 0 & 0 & 0 \end{pmatrix}. }[/math]

A matrix in Jordan normal form, similar to [math]\displaystyle{ A }[/math] is

[math]\displaystyle{ J = \begin{pmatrix} 4 & 0 & 0 & 0 \\ 0 & 5 & 1 & 0 \\ 0 & 0 & 5 & 1 \\ 0 & 0 & 0 & 5 \end{pmatrix}, }[/math]

so that [math]\displaystyle{ AM = MJ }[/math].

Applications

Matrix functions

Three of the most fundamental operations which can be performed on square matrices are matrix addition, multiplication by a scalar, and matrix multiplication. These are exactly those operations necessary for defining a polynomial function of an n × n matrix [math]\displaystyle{ A }[/math]. If we recall from basic calculus that many functions can be written as a Maclaurin series, then we can define more general functions of matrices quite easily. If [math]\displaystyle{ A }[/math] is diagonalizable, that is

[math]\displaystyle{ D = M^{-1}AM , }[/math]

with

[math]\displaystyle{ D = \begin{pmatrix} \lambda_1 & 0 & \cdots & 0 \\ 0 & \lambda_2 & \cdots & 0 \\ \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & \cdots & \lambda_n \end{pmatrix}, }[/math]

then

[math]\displaystyle{ D^k = \begin{pmatrix} \lambda_1^k & 0 & \cdots & 0 \\ 0 & \lambda_2^k & \cdots & 0 \\ \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & \cdots & \lambda_n^k \end{pmatrix} }[/math]

and the evaluation of the Maclaurin series for functions of [math]\displaystyle{ A }[/math] is greatly simplified. For example, to obtain any power k of [math]\displaystyle{ A }[/math], we need only compute [math]\displaystyle{ D^k }[/math], premultiply [math]\displaystyle{ D^k }[/math] by [math]\displaystyle{ M }[/math], and postmultiply the result by [math]\displaystyle{ M^{-1} }[/math].

Using generalized eigenvectors, we can obtain the Jordan normal form for [math]\displaystyle{ A }[/math] and these results can be generalized to a straightforward method for computing functions of nondiagonalizable matrices. (See Matrix function.)

Differential equations

Consider the problem of solving the system of linear ordinary differential equations

[math]\displaystyle{ \mathbf x' = A \mathbf x , }[/math]

(5)

where

[math]\displaystyle{ \mathbf x = \begin{pmatrix} x_1(t) \\ x_2(t) \\ \vdots \\ x_n(t) \end{pmatrix}, \quad \mathbf x' = \begin{pmatrix} x_1'(t) \\ x_2'(t) \\ \vdots \\ x_n'(t) \end{pmatrix}, }[/math]      and      [math]\displaystyle{ A = (a_{ij}) . }[/math]

If the matrix [math]\displaystyle{ A }[/math] is a diagonal matrix so that [math]\displaystyle{ a_{ij} = 0 }[/math] for [math]\displaystyle{ i \ne j }[/math], then the system (5) reduces to a system of n equations which take the form

[math]\displaystyle{ x_1' = a_{11} x_1 }[/math] [math]\displaystyle{ x_2' = a_{22} x_2 }[/math] [math]\displaystyle{ \vdots }[/math] [math]\displaystyle{ x_n' = a_{nn} x_n . }[/math]

(6)

In this case, the general solution is given by

[math]\displaystyle{ x_1 = k_1 e^{a_{11}t} }[/math]

[math]\displaystyle{ x_2 = k_2 e^{a_{22}t} }[/math]

[math]\displaystyle{ \vdots }[/math]

[math]\displaystyle{ x_n = k_n e^{a_{nn}t} . }[/math]

In the general case, we try to diagonalize [math]\displaystyle{ A }[/math] and reduce the system (5) to a system like (6) as follows. If [math]\displaystyle{ A }[/math] is diagonalizable, we have [math]\displaystyle{ D = M^{-1}AM }[/math], where [math]\displaystyle{ M }[/math] is a modal matrix for [math]\displaystyle{ A }[/math]. Substituting [math]\displaystyle{ A = MDM^{-1} }[/math], equation (5) takes the form [math]\displaystyle{ M^{-1} \mathbf x' = D(M^{-1} \mathbf x) }[/math], or

[math]\displaystyle{ \mathbf y' = D \mathbf y , }[/math]

(7)

where

[math]\displaystyle{ \mathbf x = M \mathbf y . }[/math]

(8)

The solution of (7) is

[math]\displaystyle{ y_1 = k_1 e^{\lambda_1 t} }[/math]

[math]\displaystyle{ y_2 = k_2 e^{\lambda_2 t} }[/math]

[math]\displaystyle{ \vdots }[/math]

[math]\displaystyle{ y_n = k_n e^{\lambda_n t} . }[/math]

The solution [math]\displaystyle{ \mathbf x }[/math] of (5) is then obtained using the relation (8).

On the other hand, if [math]\displaystyle{ A }[/math] is not diagonalizable, we choose [math]\displaystyle{ M }[/math] to be a generalized modal matrix for [math]\displaystyle{ A }[/math], such that [math]\displaystyle{ J = M^{-1}AM }[/math] is the Jordan normal form of [math]\displaystyle{ A }[/math]. The system [math]\displaystyle{ \mathbf y' = J \mathbf y }[/math] has the form

[math]\displaystyle{ \begin{align} y_1' & = \lambda_1 y_1 + \epsilon_1 y_2 \\ & \vdots \\ y_{n-1}' & = \lambda_{n-1} y_{n-1} + \epsilon_{n-1} y_n \\ y_n' & = \lambda_n y_n , \end{align} }[/math]

(9)

where the [math]\displaystyle{ \lambda_i }[/math] are the eigenvalues from the main diagonal of [math]\displaystyle{ J }[/math] and the [math]\displaystyle{ \epsilon_i }[/math] are the ones and zeros from the superdiagonal of [math]\displaystyle{ J }[/math]. The system (9) is often more easily solved than (5). We may solve the last equation in (9) for [math]\displaystyle{ y_n }[/math], obtaining [math]\displaystyle{ y_n = k_n e^{\lambda_n t} }[/math]. We then substitute this solution for [math]\displaystyle{ y_n }[/math] into the next to last equation in (9) and solve for [math]\displaystyle{ y_{n-1} }[/math]. Continuing this procedure, we work through (9) from the last equation to the first, solving the entire system for [math]\displaystyle{ \mathbf y }[/math]. The solution [math]\displaystyle{ \mathbf x }[/math] is then obtained using the relation (8).

Lemma:

Given the following chain of generalized eigenvectors of length [math]\displaystyle{ r, }[/math]

[math]\displaystyle{ X_1 = v_1e^{\lambda t} }[/math]

[math]\displaystyle{ X_2 = (tv_1+v_2)e^{\lambda t} }[/math]

[math]\displaystyle{ X_3 = \left(\frac{t^2}{2}v_1+tv_2+v_3\right)e^{\lambda t} }[/math]

[math]\displaystyle{ \vdots }[/math]

[math]\displaystyle{ X_r = \left(\frac{t^{r-1}}{(r-1)!}v_1+...+\frac{t^2}{2}v_{r-2}+tv_{r-1}+v_r\right)e^{\lambda t} }[/math],

these functions solve the system of equations,

[math]\displaystyle{ X' = AX. }[/math]

Proof:

Define

[math]\displaystyle{ X_j(t)=e^{\lambda t}\sum_{i = 1}^j\frac{t^{j-i}}{(j-i)!} v_i. }[/math]

Then,

[math]\displaystyle{ X'_j(t)=e^{\lambda t}\sum_{i = 1}^j\frac{t^{j-i-1}}{(j-i-1)!}v_i+e^{\lambda t}\lambda\sum_{i = 1}^j\frac{t^{j-i}}{(j-i)!}v_i }[/math].

On the other hand we have

[math]\displaystyle{ AX_j(t)=e^{\lambda t}\sum_{i = 1}^j\frac{t^{j-i}}{(j-i)!}Av_i }[/math]

[math]\displaystyle{ =e^{\lambda t}\sum_{i = 1}^j\frac{t^{j-i}}{(j-i)!}(v_{i-1}+\lambda v_i) }[/math]

[math]\displaystyle{ =e^{\lambda t}\sum_{i = 1}^j\frac{t^{j-i}}{(j-i)!}v_{i-1}+e^{\lambda t}\lambda\sum_{i = 1}^j\frac{t^{j-i}}{(j-i)!}v_i }[/math]

[math]\displaystyle{ =e^{\lambda t}\sum_{i = 1}^j\frac{t^{j-i-1}}{(j-i-1)!}v_{i}+e^{\lambda t}\lambda\sum_{i = 1}^j\frac{t^{j-i}}{(j-i)!}v_i }[/math]

[math]\displaystyle{ =X'_j(t) }[/math]

as required.

References

• Anton, Howard (1987), Elementary Linear Algebra (5th ed.), New York: John Wiley & Sons, ISBN 0-471-84819-0 
• Axler, Sheldon (1997). Linear Algebra Done Right (2nd ed.). Springer. ISBN 978-0-387-98258-8. 
• Beauregard, Raymond A.; Fraleigh, John B. (1973), A First Course In Linear Algebra: with Optional Introduction to Groups, Rings, and Fields, Boston: Houghton Mifflin Co., ISBN 0-395-14017-X, https://archive.org/details/firstcourseinlin0000beau 
• Bronson, Richard (1970), Matrix Methods: An Introduction, New York: Academic Press 
• Burden, Richard L.; Faires, J. Douglas (1993), Numerical Analysis (5th ed.), Boston: Prindle, Weber and Schmidt, ISBN 0-534-93219-3, https://archive.org/details/numericalanalysi00burd 
• Cullen, Charles G. (1966), Matrices and Linear Transformations, Reading: Addison-Wesley 
• Franklin, Joel N. (1968), Matrix Theory, Englewood Cliffs: Prentice-Hall 
• Golub, Gene H.; Van Loan, Charles F. (1996), Matrix Computations (3rd ed.), Baltimore: Johns Hopkins University Press, ISBN 0-8018-5414-8 
• Harper, Charlie (1976), Introduction to Mathematical Physics, New Jersey: Prentice-Hall, ISBN 0-13-487538-9 
• Herstein, I. N. (1964), Topics In Algebra, Waltham: Blaisdell Publishing Company, ISBN 978-1114541016 
• Kreyszig, Erwin (1972), Advanced Engineering Mathematics (3rd ed.), New York: John Wiley & Sons, ISBN 0-471-50728-8, https://archive.org/details/advancedengineer00krey 
• Nering, Evar D. (1970), Linear Algebra and Matrix Theory (2nd ed.), New York: John Wiley & Sons