Invariant subspace problem

From HandWiki
Short description: Partially unsolved problem in mathematics
The vector [math]\displaystyle{ x }[/math] is an eigenvector of the matrix [math]\displaystyle{ A }[/math]. Every operator on a non-trivial complex finite dimensional vector space has an eigenvector, solving the invariant subspace problem for these spaces.

In the field of mathematics known as functional analysis, the invariant subspace problem is a partially unresolved problem asking whether every bounded operator on a complex Banach space sends some non-trivial closed subspace to itself. Many variants of the problem have been solved, by restricting the class of bounded operators considered or by specifying a particular class of Banach spaces. The problem is still open for separable Hilbert spaces (in other words, each example, found so far, of an operator with no non-trivial invariant subspaces is an operator that acts on a Banach space that is not isomorphic to a separable Hilbert space).


The problem seems to have been stated in the mid-1900s after work by Beurling and von Neumann,[1] who found (but never published) a positive solution for the case of compact operators. It was then posed by Paul Halmos for the case of operators [math]\displaystyle{ T }[/math] such that [math]\displaystyle{ T^2 }[/math] is compact. This was resolved affirmatively, for the more general class of polynomially compact operators (operators [math]\displaystyle{ T }[/math] such that [math]\displaystyle{ p(T) }[/math] is a compact operator for a suitably chosen non-zero polynomial [math]\displaystyle{ p }[/math]), by Allen R. Bernstein and Abraham Robinson in 1966 (see Non-standard analysis § Invariant subspace problem for a summary of the proof).

For Banach spaces, the first example of an operator without an invariant subspace was constructed by Per Enflo. He proposed a counterexample to the invariant subspace problem in 1975, publishing an outline in 1976. Enflo submitted the full article in 1981 and the article's complexity and length delayed its publication to 1987[2] Enflo's long "manuscript had a world-wide circulation among mathematicians"[1] and some of its ideas were described in publications besides Enflo (1976).[3] Enflo's works inspired a similar construction of an operator without an invariant subspace for example by Beauzamy, who acknowledged Enflo's ideas.[2]

In the 1990s, Enflo developed a "constructive" approach to the invariant subspace problem on Hilbert spaces.[4]

In May 2023, a preprint of Enflo appeared on arXiv,[5] which, if correct, solves the problem for Hilbert spaces and completes the picture.

Precise statement

Formally, the invariant subspace problem for a complex Banach space [math]\displaystyle{ H }[/math] of dimension > 1 is the question whether every bounded linear operator [math]\displaystyle{ T: H \to H }[/math] has a non-trivial closed [math]\displaystyle{ T }[/math]-invariant subspace: a closed linear subspace [math]\displaystyle{ W }[/math] of [math]\displaystyle{ H }[/math], which is different from [math]\displaystyle{ \{0\} }[/math] and from [math]\displaystyle{ H }[/math], such that [math]\displaystyle{ T(W)\subset W }[/math].

A negative answer to the problem is closely related to properties of the orbits [math]\displaystyle{ T }[/math]. If [math]\displaystyle{ x }[/math] is an element of the Banach space [math]\displaystyle{ H }[/math], the orbit of [math]\displaystyle{ x }[/math] under the action of [math]\displaystyle{ T }[/math], denoted by [math]\displaystyle{ [x] }[/math], is the subspace generated by the sequence [math]\displaystyle{ \{ T^{n}(x)\,:\, n \ge 0\} }[/math]. This is also called the [math]\displaystyle{ T }[/math]-cyclic subspace generated by [math]\displaystyle{ x }[/math]. From the definition it follows that [math]\displaystyle{ [x] }[/math] is a [math]\displaystyle{ T }[/math]-invariant subspace. Moreover, it is the minimal [math]\displaystyle{ T }[/math]-invariant subspace containing [math]\displaystyle{ x }[/math]: if [math]\displaystyle{ W }[/math] is another invariant subspace containing [math]\displaystyle{ x }[/math], then necessarily [math]\displaystyle{ T^n(x) \in W }[/math] for all [math]\displaystyle{ n \ge 0 }[/math] (since [math]\displaystyle{ W }[/math] is [math]\displaystyle{ T }[/math]-invariant), and so [math]\displaystyle{ [x]\subset W }[/math]. If [math]\displaystyle{ x }[/math] is non-zero, then [math]\displaystyle{ [x] }[/math] is not equal to [math]\displaystyle{ \{0\} }[/math], so its closure is either the whole space [math]\displaystyle{ H }[/math] (in which case [math]\displaystyle{ x }[/math] is said to be a cyclic vector for [math]\displaystyle{ T }[/math]) or it is a non-trivial [math]\displaystyle{ T }[/math]-invariant subspace. Therefore, a counterexample to the invariant subspace problem would be a Banach space [math]\displaystyle{ H }[/math] and a bounded operator [math]\displaystyle{ T: H \to H }[/math] for which every non-zero vector [math]\displaystyle{ x\in H }[/math] is a cyclic vector for [math]\displaystyle{ T }[/math]. (Where a "cyclic vector" [math]\displaystyle{ x }[/math] for an operator [math]\displaystyle{ T }[/math] on a Banach space [math]\displaystyle{ H }[/math] means one for which the orbit [math]\displaystyle{ [x] }[/math] of [math]\displaystyle{ x }[/math] is dense in [math]\displaystyle{ H }[/math].)

Known special cases

While the case of the invariant subspace problem for separable Hilbert spaces is still open, several other cases have been settled for topological vector spaces (over the field of complex numbers):

  • For finite-dimensional complex vector spaces, every operator admits an eigenvector, so it has a 1-dimensional invariant subspace.
  • The conjecture is true if the Hilbert space [math]\displaystyle{ H }[/math] is not separable (i.e. if it has an uncountable orthonormal basis). In fact, if [math]\displaystyle{ x }[/math] is a non-zero vector in [math]\displaystyle{ H }[/math], the norm closure of the linear orbit [math]\displaystyle{ [x] }[/math] is separable (by construction) and hence a proper subspace and also invariant.
  • von Neumann showed[6] that any compact operator on a Hilbert space of dimension at least 2 has a non-trivial invariant subspace.
  • The spectral theorem shows that all normal operators admit invariant subspaces.
  • (Aronszajn Smith) proved that every compact operator on any Banach space of dimension at least 2 has an invariant subspace.
  • (Bernstein Robinson) proved using non-standard analysis that if the operator [math]\displaystyle{ T }[/math] on a Hilbert space is polynomially compact (in other words [math]\displaystyle{ p(T) }[/math] is compact for some non-zero polynomial [math]\displaystyle{ p }[/math]) then [math]\displaystyle{ T }[/math] has an invariant subspace. Their proof uses the original idea of embedding the infinite-dimensional Hilbert space in a hyperfinite-dimensional Hilbert space (see Non-standard analysis).
  • (Halmos 1966), after having seen Robinson's preprint, eliminated the non-standard analysis from it and provided a shorter proof in the same issue of the same journal.
  • (Lomonosov 1973) gave a very short proof using the Schauder fixed point theorem that if the operator [math]\displaystyle{ T }[/math] on a Banach space commutes with a non-zero compact operator then [math]\displaystyle{ T }[/math] has a non-trivial invariant subspace. This includes the case of polynomially compact operators because an operator commutes with any polynomial in itself. More generally, he showed that if [math]\displaystyle{ S }[/math] commutes with a non-scalar operator [math]\displaystyle{ T }[/math] that commutes with a non-zero compact operator, then [math]\displaystyle{ S }[/math] has an invariant subspace.[7]
  • The first example of an operator on a Banach space with no non-trivial invariant subspaces was found by Per Enflo (1976, 1987), and his example was simplified by (Beauzamy 1985).
  • The first counterexample on a "classical" Banach space was found by Charles Read (1984, 1985), who described an operator on the classical Banach space [math]\displaystyle{ l_1 }[/math] with no invariant subspaces.
  • Later Charles Read (1988) constructed an operator on [math]\displaystyle{ l_1 }[/math] without even a non-trivial closed invariant subset, that is that for every vector [math]\displaystyle{ x }[/math] the set [math]\displaystyle{ \{ T^{n}(x)\,:\, n \ge 0\} }[/math] is dense, in which case the vector is called hypercyclic (the difference with the case of cyclic vectors is that we are not taking the subspace generated by the points [math]\displaystyle{ \{ T^{n}(x)\,:\, n \ge 0\} }[/math] in this case).
  • (Atzmon 1983) gave an example of an operator without invariant subspaces on a nuclear Fréchet space.
  • (Śliwa 2008) proved that any infinite dimensional Banach space of countable type over a non-Archimedean field admits a bounded linear operator without a non-trivial closed invariant subspace. This completely solves the non-Archimedean version of this problem, posed by van Rooij and Schikhof in 1992.
  • (Argyros Haydon) gave the construction of an infinite-dimensional Banach space such that every continuous operator is the sum of a compact operator and a scalar operator, so in particular every operator has an invariant subspace.


  1. 1.0 1.1 Yadav (2005), p. 292.
  2. 2.0 2.1 Beauzamy (1988); Yadav (2005).
  3. See, for example, Radjavi & Rosenthal (1982).
  4. Page 401 in Foiaş, Ciprian; Jung, Il Bong; Ko, Eungil; Pearcy, Carl (2005). "On quasinilpotent operators. III". Journal of Operator Theory 54 (2): 401–414. . Enflo's method of ("forward") "minimal vectors" is also noted in the review of this research article by Gilles Cassier in Mathematical Reviews: MR2186363
  5. Enflo, Per H. (May 26, 2023). "On the invariant subspace problem in Hilbert spaces". arXiv:2305.15442 [math.FA].
  6. Von Neumann's proof was never published, as relayed in a private communication to the authors of (Aronszajn Smith). A version of that proof, independently discovered by Aronszajn, is included at the end of that paper.
  7. See (Pearcy Shields) for a review.