Numerical range

In the mathematical field of linear algebra and convex analysis, the numerical range or field of values or Wertvorrat or Wertevorrat of a complex $n\times n$ matrix A is the set

$W(A)=\{\frac{{\mathbf{𝐱}}^{*}A\mathbf{𝐱}}{{\mathbf{𝐱}}^{*}\mathbf{𝐱}}\mid \mathbf{𝐱}\in {\mathbb{ℂ}}^n,\mathbf{𝐱}=0\}=\{⟨\mathbf{𝐱},A\mathbf{𝐱}⟩\mid \mathbf{𝐱}\in {\mathbb{ℂ}}^n,\Vert \mathbf{𝐱}{\Vert }_2=1\}$

where ${\mathbf{𝐱}}^{*}$ denotes the conjugate transpose of the vector $\mathbf{𝐱}$. The numerical range includes, in particular, the diagonal entries of the matrix (obtained by choosing x equal to the unit vectors along the coordinate axes) and the eigenvalues of the matrix (obtained by choosing x equal to the eigenvectors).

Equivalently, the elements of $W(A)$ are of the form $\mathrm{tr}(AP)$, where $P$ is a Hermitian projection operator from ${\mathbb{ℂ}}^2$ to a one-dimensional subspace.

In engineering, numerical ranges are used as a rough estimate of eigenvalues of A. Recently, generalizations of the numerical range are used to study quantum computing.

A related concept is the numerical radius, which is the largest absolute value of the numbers in the numerical range, i.e.

$r(A)=\sup \{|\lambda |:\lambda \in W(A)\}=\underset{\Vert x{\Vert }_2=1}{\sup }|⟨\mathbf{𝐱},A\mathbf{𝐱}⟩|.$

Properties

Let sum of sets denote a sumset.

General properties

1. The numerical range is the range of the Rayleigh quotient.
2. (Hausdorff–Toeplitz theorem) The numerical range is convex and compact.
3. $W(\alpha A+\beta I)=\alpha W(A)+\{\beta \}$ for all square matrix $A$ and complex numbers $\alpha$ and $\beta$. Here $I$ is the identity matrix.
4. $W(A)$ is a subset of the closed right half-plane if and only if $A+A^{*}$ is positive semidefinite.
5. The numerical range $W(\cdot )$ is the only function on the set of square matrices that satisfies (2), (3) and (4).
6. $W(UA{U}^{*})=W(A)$ for any unitary $U$.
7. $W(A^{*})=W(A{)}^{*}$.
8. If $A$ is Hermitian, then $W(A)$ is on the real line. If $A$ is anti-Hermitian, then $W(A)$ is on the imaginary line.
9. $W(A)=\{z\}$ if and only if $A=zI$.
10. (Sub-additive) $W(A+B)\subseteq W(A)+W(B)$.
11. $W(A)$ contains all the eigenvalues of $A$.
12. The numerical range of a $2\times 2$ matrix is a filled ellipse.
13. $W(A)$ is a real line segment $[\alpha ,\beta ]$ if and only if $A$ is a Hermitian matrix with its smallest and the largest eigenvalues being $\alpha$ and $\beta$.

Normal matrices

1. If $A$ is normal, and $x\in \mathrm{span}(v_1,\dots ,v_k)$, where $v_1,\dots ,v_k$ are eigenvectors of $A$ corresponding to ${\lambda }_1,\dots ,{\lambda }_k$, respectively, then $⟨x,Ax⟩\in \mathrm{hull}({\lambda }_1,\dots ,{\lambda }_k)$.
2. If $A$ is a normal matrix then $W(A)$ is the convex hull of its eigenvalues.
3. If $\alpha$ is a sharp point on the boundary of $W(A)$, then $\alpha$ is a normal eigenvalue of $A$.

Numerical radius

1. $r(\cdot )$ is a unitarily invariant norm on the space of $n\times n$ matrices.
2. $r(A)\le \Vert A{\Vert }_{\mathrm{op}}\le 2r(A)$, where $\Vert \cdot {\Vert }_{\mathrm{op}}$ denotes the operator norm.

Proof of (12)

The elements of $W(A)$ are of the form $\mathrm{tr}(AP)$, where $P$ is projection from ${\mathbb{ℂ}}^2$ to a one-dimensional subspace.

The space of all one-dimensional subspaces of ${\mathbb{ℂ}}^2$ is $\mathbb{\mathbb{P}}{\mathbb{ℂ}}^1$, which is a 2-sphere. The image of a 2-sphere under a linear projection is a filled ellipse.

In more detail, such $P$ are of the form $$\frac{1}{2}I+\frac{1}{2}\left[\begin{array}{cc}\cos 2\theta & e^{i\varphi }\sin 2\theta \\ e^{-i\varphi }\sin 2\theta & -\cos 2\theta \end{array}\right]=\frac{1}{2}\left[\begin{array}{cc}1+z& x+iy\\ x-iy& 1-z\end{array}\right]$$ where $x,y,z$, satisfying $x^2+y^2+z^2=1$, is a point on the unit 2-sphere.

Therefore, the elements of $W(A)$, regarded as elements of ${\mathbb{ℝ}}^2$ is the composition of two real linear maps $(x,y,z)\mapsto \frac{1}{2}\left[\begin{array}{cc}1+z& x+iy\\ x-iy& 1-z\end{array}\right]$ and $M\mapsto \mathrm{tr}(AM)$, which maps the 2-sphere to a filled ellipse.

Proof of (2)

$W(A)$ is the image of a continuous map $x\mapsto ⟨x,Ax⟩$ from the ${\mathbb{\mathbb{P}}\mathbb{ℂ}}^n$, so it is compact.

Given two complex nonzero vectors $x,y$, let $P_x,P_y$ be their corresponding Hermitian projectors from ${\mathbb{ℂ}}^n$ to their respective spans. Let $P$ be the Hermitian projector to the span of both. We have that $P^{*}AP$ is an operator on $\mathrm{Span}(x,y)$.

Therefore, the “restricted numerical range” of $P^{*}AP$, defined by $\{\mathrm{Tr}(P^{*}AP{P}_z):z\in \mathrm{Span}(x,y),z\ne 0\}$, is a closed ellipse, according to (12). It is also the case that if $z\in \mathrm{Span}(x,y)$ is nonzero, then $\mathrm{Tr}(P^{*}AP{P}_z)=\mathrm{Tr}(AP{P}_{z}P)=\mathrm{Tr}(A{P}_z)\in W(A)$. Therefore, the restricted numerical range is contained in the full numerical range of $A$.

Thus, if $W(A)$ contains $\mathrm{Tr}(A{P}_x),\mathrm{Tr}(A{P}_y)$, then it contains a closed ellipse that also contains $\mathrm{Tr}(A{P}_x),\mathrm{Tr}(A{P}_y)$, so it contains the line segment between them.

Proof of (5)

Let $W$ satisfy these properties. Let $W_0$ be the original numerical range.

Fix some matrix $A$. We show that the supporting planes of $W(A)$ and $W_0(A)$ are identical. This would then imply that $W(A)=W_0(A)$ since they are both convex and compact.

By property (4), $W(A)$ is nonempty. Let $z$ be a point on the boundary of $W(A)$, then we can translate and rotate the complex plane so that the point translates to the origin, and the region $W(A)$ falls entirely within ${\mathbb{ℂ}}^{+}$. That is, for some $\varphi \in \mathbb{ℝ}$, the set $e^{i\varphi }(W(A)-z)$ lies entirely within ${\mathbb{ℂ}}^{+}$, while for any $t>0$, the set $e^{i\varphi }(W(A)-z)-tI$ does not lie entirely in ${\mathbb{ℂ}}^{+}$.

The two properties of $W$ then imply that $$e^{i\varphi }(A-z)+e^{-i\varphi }(A-z{)}^{*}⪰0$$ and that inequality is sharp, meaning that $e^{i\varphi }(A-z)+e^{-i\varphi }(A-z{)}^{*}$ has a zero eigenvalue. This is a complete characterization of the supporting planes of $W(A)$.

The same argument applies to $W_0(A)$, so they have the same supporting planes.

Normal matrices

Proof of (1), (2)

For (2), if $A$ is normal, then it has a full eigenbasis, so it reduces to (1).

Since $A$ is normal, by the spectral theorem, there exists a unitary matrix $U$ such that $A=UD{U}^{*}$, where $D$ is a diagonal matrix containing the eigenvalues ${\lambda }_1,{\lambda }_2,\dots ,{\lambda }_n$ of $A$.

Let $x=c_1{v}_1+c_2{v}_2+\cdots +c_k{v}_k$. Using the linearity of the inner product, that $A{v}_j={\lambda }_j{v}_j$, and that $\left\{v_i\right\}$ are orthonormal, we have:

$$⟨x,Ax⟩=\sum _{i,j=1}^k{c}_i^{*}{c}_j⟨v_i,{\lambda }_j{v}_j⟩=\sum _{i=1}^k{\left|c_i\right|}^2{\lambda }_i\in \mathrm{hull}({\lambda }_1,\dots ,{\lambda }_k)$$

Proof (3)

By affineness of $W$, we can translate and rotate the complex plane, so that we reduce to the case where $\partial W(A)$ has a sharp point at $0$, and that the two supporting planes at that point both make an angle ${\varphi }_1,{\varphi }_2$ with the imaginary axis, such that ${\varphi }_1<{\varphi }_2,e^{i{\varphi }_1}\ne e^{i{\varphi }_2}$ since the point is sharp.

Since $0\in W(A)$, there exists a unit vector $x_0$ such that $x_0^{*}A{x}_0=0$.

By general property (4), the numerical range lies in the sectors defined by: $$\mathrm{Re}(e^{i\theta }⟨x,Ax⟩)\ge 0\text{for all }\theta \in [{\varphi }_1,{\varphi }_2]\text{ and nonzero }x\in {\mathbb{ℂ}}^n.$$ At $x=x_0$, the directional derivative in any direction $y$ must vanish to maintain non-negativity. Specifically:
$${\frac{d}{dt}\mathrm{Re}(e^{i\theta }⟨x_0+ty,A(x_0+ty)⟩)|}_{t=0}=0\mathrm{\forall }y\in {\mathbb{ℂ}}^n,\theta \in [{\varphi }_1,{\varphi }_2].$$ Expanding this derivative:
$$\mathrm{Re}\left(e^{i\theta }(⟨y,A{x}_0⟩+⟨x_0,Ay⟩)\right)=0\mathrm{\forall }y\in {\mathbb{ℂ}}^n,\theta \in [{\varphi }_1,{\varphi }_2].$$

Since the above holds for all $\theta \in [{\varphi }_1,{\varphi }_2]$, we must have: $$⟨y,A{x}_0⟩+⟨x_0,Ay⟩=0\mathrm{\forall }y\in {\mathbb{ℂ}}^n.$$

For any $y\in {\mathbb{ℂ}}^n$ and $\alpha \in \mathbb{ℂ}$, substitute $\alpha y$ into the equation: $$\alpha ⟨y,A{x}_0⟩+{\alpha }^{*}⟨x_0,Ay⟩=0.$$ Choose $\alpha =1$ and $\alpha =i$, then simplify, we obtain $⟨y,A{x}_0⟩=0$ for all $y$, thus $A{x}_0=0$.

Numerical radius

Proof of (2)

Let $v=\arg \underset{\Vert x{\Vert }_2=1}{\max }|⟨x,Ax⟩|$. We have $r(A)=|⟨v,Av⟩|$.

By Cauchy–Schwarz, $$|⟨v,Av⟩|\le \Vert v{\Vert }_2\Vert Av{\Vert }_2=\Vert Av{\Vert }_2\le \Vert A{\Vert }_{op}$$

For the other one, let $A=B+iC$, where $B,C$ are Hermitian. $$\Vert A{\Vert }_{op}\le \Vert B{\Vert }_{op}+\Vert C{\Vert }_{op}$$

Since $W(B)$ is on the real line, and $W(iC)$ is on the imaginary line, the extremal points of $W(B),W(iC)$ appear in $W(A)$, shifted, thus both $\Vert B{\Vert }_{op}=r(B)\le r(A),\Vert C{\Vert }_{op}=r(iC)\le r(A)$.

Generalisations

• C-numerical range
• Joint numerical range
• Product numerical range
• Polynomial numerical hull

Higher-rank numerical range

The numerical range is equivalent to the following definition:$$W(A)=\{\lambda \in \mathbb{ℂ}:PMP=\lambda P\text{ for some Hermitian projector }P\text{ of rank }1\}$$This allows a generalization to higher-rank numerical ranges, one for each $k=1,2,3,\dots$:$$W_k(A)=\{\lambda \in \mathbb{ℂ}:PMP=\lambda P\text{ for some Hermitian projector }P\text{ of rank }k\}$$$W_k(A)$ is always closed and convex, but it might be empty. It is guaranteed to be nonempty if $k<n/3+1$, and there exists some $A$ such that $W_k(A)$ is empty if $k\ge n/3+1$.

See also

• Spectral theory
• Rayleigh quotient
• Workshop on Numerical Ranges and Numerical Radii

Bibliography

Books

• Bonsall, F.F.; Duncan, J. (1971), Numerical Ranges of Operators on Normed Spaces and of Elements of Normed Algebras, Cambridge University Press, ISBN 978-0-521-07988-4 
• Bonsall, F.F.; Duncan, J. (1971), Numerical Ranges II, Cambridge University Press, ISBN 978-0-521-20227-5 
• Horn, Roger A.; Johnson, Charles R. (1991), Topics in Matrix Analysis, Cambridge University Press, Chapter 1, ISBN 978-0-521-46713-1 .
• Horn, Roger A.; Johnson, Charles R. (1990), Matrix Analysis, Cambridge University Press, Ch. 5.7, ex. 21, ISBN 0-521-30586-1 
• Bhatia, Rajendra (1997). Matrix analysis. Graduate texts in mathematics. New York Berlin Heidelberg: Springer. ISBN 978-0-387-94846-1. 
• Gustafson, Karl E.; Rao, Duggirala K. M. (1997). Numerical Range: The Field of Values of Linear Operators and Matrices. Universitext. New York, NY: Springer. doi:10.1007/978-1-4613-8498-4. ISBN 978-0-387-94835-5. 

Papers

• Toeplitz, Otto (1918). "Das algebraische Analogon zu einem Satze von Fejér" (in de). Mathematische Zeitschrift 2 (1-2): 187–197. doi:10.1007/BF01212904. ISSN 0025-5874. https://zenodo.org/records/2474150/files/article.pdf. 
• Hausdorff, Felix (1919). "Der Wertvorrat einer Bilinearform" (in de). Mathematische Zeitschrift 3 (1): 314–316. doi:10.1007/BF01292610. ISSN 0025-5874. 
• Choi, M.D.; Kribs, D.W.; Życzkowski (2006), "Quantum error correcting codes from the compression formalism", Rep. Math. Phys. 58 (1): 77–91, doi:10.1016/S0034-4877(06)80041-8, Bibcode: 2006RpMP...58...77C .
• Dirr, G.; Helmkel, U.; Kleinsteuber, M.; Schulte-Herbrüggen, Th. (2006), "A new type of C-numerical range arising in quantum computing", Proc. Appl. Math. Mech. 6: 711–712, doi:10.1002/pamm.200610336 
• Li, C.K. (1996), "A simple proof of the elliptical range theorem", Proc. Am. Math. Soc. 124 (7): 1985, doi:10.1090/S0002-9939-96-03307-2 .
• Keeler, Dennis S.; Rodman, Leiba; Spitkovsky, Ilya M. (1997), "The numerical range of 3 × 3 matrices", Linear Algebra and Its Applications 252 (1–3): 115, doi:10.1016/0024-3795(95)00674-5 .
• Johnson, Charles R. (1976). "Functional characterizations of the field of values and the convex hull of the spectrum". Proceedings of the American Mathematical Society (American Mathematical Society (AMS)) 61 (2): 201–204. doi:10.1090/s0002-9939-1976-0437555-3. ISSN 0002-9939. https://www.ams.org/proc/1976-061-02/S0002-9939-1976-0437555-3/S0002-9939-1976-0437555-3.pdf.