View source for Gateaux derivative

{{Short description|Generalization of the concept of directional derivative}}
{{Calculus}}
In [[Mathematics|mathematics]], the '''Gateaux differential''' or '''Gateaux derivative''' is a generalization of the concept of [[Directional derivative|directional derivative]] in [[Differential calculus|differential calculus]]. Named after René Gateaux, a French mathematician who died at age 25 in [[Social:World War I|World War I]], it is defined for functions between [[Locally convex topological vector space|locally convex]] [[Topological vector space|topological vector space]]s such as [[Banach space]]s.  Like the [[Fréchet derivative]] on a Banach space, the Gateaux differential is often used to formalize the [[Functional derivative|functional derivative]] commonly used in the [[Calculus of variations|calculus of variations]] and [[Physics:Physics|physics]].

Unlike other forms of derivatives, the Gateaux differential of a function may be nonlinear. However, often the definition of the Gateaux differential also requires that it be a continuous linear transformation.  Some authors, such as {{harvtxt|Tikhomirov|2001}}, draw a further distinction between the Gateaux differential (which may be nonlinear) and the Gateaux derivative (which they take to be linear).  In most applications, continuous linearity follows from some more primitive condition which is natural to the particular setting, such as imposing [[Holomorphic function|complex differentiability]] in the context of infinite dimensional holomorphy or continuous differentiability in nonlinear analysis.

==Definition==

Suppose <math>X</math> and <math>Y</math> are [[Locally convex topological vector space|locally convex]] [[Topological vector space|topological vector space]]s (for example,  [[Banach space]]s), <math>U \subseteq X</math> is open, and <math>F : U \to Y.</math>  The Gateaux differential <math>dF(u; \psi)</math> of <math>F</math> at <math>u \in U</math> in the direction <math>\psi \in X</math> is defined as

{{NumBlk|:|<math>dF(u;\psi) = \lim_{\tau\to 0} \frac{F(u+\tau \psi)-F(u)}{\tau} = \left.\frac{d}{d\tau}F(u+\tau \psi)\right|_{\tau=0}</math>|{{EquationRef|1}}}}

If the limit exists for all <math>\psi \in X,</math> then one says that <math>F</math> is Gateaux differentiable at <math>u.</math>

The limit appearing in ({{EquationNote|1}}) is taken relative to the topology of <math>Y.</math>  If <math>X</math> and <math>Y</math> are real topological vector spaces, then the limit is taken for real <math>\tau.</math> On the other hand, if <math>X</math> and <math>Y</math> are complex topological vector spaces, then the limit above is usually taken as <math>\tau \to 0</math> in the [[Complex plane|complex plane]] as in the definition of [[Holomorphic function|complex differentiability]].  In some cases, a [[Weak topology|weak limit]] is taken instead of a strong limit, which leads to the notion of a weak Gateaux derivative.

==Linearity and continuity==

At each point <math>u \in U,</math> the Gateaux differential defines a function
<math display=block>dF(u;\cdot) : X \to Y.</math>

This function is homogeneous in the sense that for all scalars <math>\alpha,</math>
<math display=block>dF(u;\alpha\psi) = \alpha dF(u;\psi).\,</math>

However, this function need not be additive, so that the Gateaux differential may fail to be linear, unlike the [[Fréchet derivative]].  Even if linear, it may fail to depend continuously on <math>\psi</math> if <math>X</math> and <math>Y</math> are infinite dimensional.  Furthermore, for Gateaux differentials that {{em|are}} linear and continuous in <math>\psi,</math> there are several inequivalent ways to formulate their continuous differentiability.

For example, consider the real-valued function <math>F</math> of two real variables defined by
<math display=block>F(x, y) =
\begin{cases}
\dfrac{x^3}{x^2+y^2} & \text{if } (x, y) \neq (0, 0), \\
0 & \text{if } (x, y) = (0, 0).
\end{cases}</math>
This is Gateaux differentiable at <math>(0, 0).</math> with its differential there being
<math display=block>dF(0,0; a, b) =
\begin{cases}\dfrac{F(\tau a,\tau b)-0}{\tau} & (a,b) \neq (0,0), \\
0 & (a,b) = (0,0)\end{cases}
=
\begin{cases}\dfrac{a^3}{a^2+b^2} & (a,b) \neq (0,0), \\
0 & (a,b) = (0,0).\end{cases}</math>
However this is continuous but not linear in the arguments <math>(a, b).</math>  In infinite dimensions, any discontinuous linear functional on <math>X</math> is Gateaux differentiable, but its Gateaux differential at <math>0</math> is linear but not continuous.

'''Relation with the Fréchet derivative'''

If <math>F</math> is Fréchet differentiable, then it is also Gateaux differentiable, and its Fréchet and Gateaux derivatives agree.  The converse is clearly not true, since the Gateaux derivative may fail to be linear or continuous.  In fact, it is even possible for the Gateaux derivative to be linear and continuous but for the Fréchet derivative to fail to exist.

Nevertheless, for functions <math>F</math> from a {{em|complex}} Banach space <math>X</math> to another complex Banach space <math>Y,</math> the Gateaux derivative (where the limit is taken over complex <math>\tau</math> tending to zero as in the definition of [[Holomorphic function|complex differentiability]]) is automatically linear, a theorem of {{harvtxt|Zorn|1945}}.  Furthermore, if <math>F</math> is (complex) Gateaux differentiable at each <math>u \in U</math> with derivative
<math display=block>DF(u) : \psi \mapsto dF(u;\psi)</math>
then <math>F</math> is Fréchet differentiable on <math>U</math> with Fréchet derivative <math>DF</math> {{harv|Zorn|1946}}. This is analogous to the result from basic [[Complex analysis|complex analysis]] that a function is [[Analytic function|analytic]] if it is complex differentiable in an open set, and is a fundamental result in the study of infinite dimensional holomorphy.

'''Continuous differentiability'''

Continuous Gateaux differentiability may be defined in two inequivalent ways. Suppose that <math>F : U \to Y</math> is Gateaux differentiable at each point of the open set <math>U.</math>  One notion of continuous differentiability in <math>U</math> requires that the mapping on the product space
<math display=block>dF : U \times X \to Y \,</math>
be continuous.  Linearity need not be assumed: if <math>X</math> and <math>Y</math> are Fréchet spaces, then <math>dF(u; \cdot)</math> is automatically bounded and linear for all <math>u</math> {{harv|Hamilton|1982}}.

A stronger notion of continuous differentiability requires that
<math display=block>u \mapsto DF(u) \,</math>
be a continuous mapping
<math display=block>U \to L(X,Y) \,</math>
from <math>U</math> to the space of continuous linear functions from <math>X</math> to <math>Y.</math>  Note that this already presupposes the linearity of <math>DF(u).</math>

As a matter of technical convenience, this latter notion of continuous differentiability is typical (but not universal) when the spaces <math>X</math> and <math>Y</math> are Banach, since <math>L(X, Y)</math> is also Banach and standard results from functional analysis can then be employed.  The former is the more common definition in areas of nonlinear analysis where the function spaces involved are not necessarily Banach spaces.  For instance, [[Differentiation in Fréchet spaces|differentiation in Fréchet spaces]] has applications such as the Nash–Moser inverse function theorem in which the function spaces of interest often consist of smooth functions on a [[Differentiable manifold|manifold]].

==Higher derivatives==

Whereas higher order Fréchet derivatives are naturally defined as multilinear functions by iteration, using the isomorphisms <math>L^n(X, Y) = L(X, L^{n - 1}(X, Y)),</math> higher order Gateaux derivative cannot be defined in this way.  Instead the <math>n</math>th order Gateaux derivative of a function <math>F : U \subseteq X \to Y</math> in the direction <math>h</math> is defined by
{{NumBlk|:|<math>d^nF(u;h) = \left.\frac{d^n}{d\tau^n}F(u+\tau h)\right|_{\tau=0}.</math>|{{EquationRef|2}}}}
Rather than a multilinear function, this is instead a [[Homogeneous function|homogeneous function]] of degree <math>n</math> in <math>h.</math>

There is another candidate for the definition of the higher order derivative, the function
{{NumBlk|:|<math>D^2F(u)\{h,k\} = \lim_{\tau\to 0} \frac{DF(u+\tau k)h - DF(u)h}{\tau} = \left.\frac{\partial^2}{\partial\tau \, \partial\sigma}F(u+\sigma h + \tau k)\right|_{\tau=\sigma=0}</math>|{{EquationRef|3}}}}
that arises naturally in the calculus of variations as the second variation of <math>F,</math> at least in the special case where <math>F</math> is scalar-valued.  However, this may fail to have any reasonable properties at all, aside from being separately homogeneous in <math>h</math> and <math>k.</math>  It is desirable to have sufficient conditions in place to ensure that <math>D^2 F(u)\{h, k\}</math> is a symmetric bilinear function of <math>h</math> and <math>k,</math> and that it agrees with the [[Polarization of an algebraic form|polarization]] of <math>d^n F.</math>

For instance, the following sufficient condition holds {{harv|Hamilton|1982}}.  Suppose that <math>F</math> is <math>C^1</math> in the sense that the mapping
<math display=block>DF : U\times X\to Y</math>
is continuous in the [[Product topology|product topology]], and moreover that the second derivative defined by ({{EquationNote|3}}) is also continuous in the sense that
<math display=block>D^2F : U\times X\times X\to Y</math>
is continuous.  Then <math>D^2 F(u)\{h, k\}</math> is bilinear and symmetric in <math>h</math> and <math>k.</math> By virtue of the bilinearity, the polarization identity holds 
<math display=block>D^2F(u)\{h,k\} = \frac{1}{2}d^2F(u;h+k)-d^2F(u;h)-d^2F(u;k)</math>
relating the second order derivative <math>D^2 F(u)</math> with the differential <math>d^2 F(u; -).</math>  Similar conclusions hold for higher order derivatives.

==Properties==

A version of the [[Fundamental theorem of calculus|fundamental theorem of calculus]] holds for the Gateaux derivative of <math>F,</math> provided <math>F</math> is assumed to be sufficiently continuously differentiable.  Specifically:

* Suppose that <math>F : X \to Y</math> is <math>C^1</math> in the sense that the Gateaux derivative is a continuous function <math>dF : U \times X \to Y.</math>  Then for any <math>u \in U</math> and <math>h \in X,</math><math display=block>F(u+h) - F(u) = \int_0^1 dF(u+th;h)\,dt</math> where the integral is the [[Pettis integral|Gelfand–Pettis integral]] (the weak integral) ({{harvtxt|Vainberg|1964}}).

Many of the other familiar properties of the derivative follow from this, such as multilinearity and commutativity of the higher-order derivatives.  Further properties, also consequences of the fundamental theorem, include:

* ('''The [[Chain rule|chain rule]]''')<br/><math display=block>d(G\circ F)(u;x) = dG(F(u); dF(u;x))</math> for all <math>u \in U</math> and <math>x \in X.</math>  (Importantly, as with simple [[Partial derivative|partial derivative]]s, the Gateaux derivative does {{em|not}} satisfy the chain rule if the derivative is permitted to be discontinuous.)

* ('''[[Taylor's theorem]] with remainder''')<br/>Suppose that the line segment between <math>u \in U</math> and <math>u + h</math> lies entirely within <math>U.</math>  If <math>F</math> is <math>C^k</math> then <math display=block>F(u+h)=F(u)+dF(u;h)+\frac{1}{2!}d^2F(u;h)+\dots+\frac{1}{(k-1)!}d^{k-1}F(u;h)+R_k</math> where the remainder term is given by <math display=block>R_k(u;h)=\frac{1}{(k-1)!}\int_0^1(1-t)^{k-1}d^kF(u+th;h)\,dt</math>

==Example==

Let <math>X</math> be the [[Hilbert space]] of [[Square-integrable function|square-integrable function]]s on a [[Lebesgue measure|Lebesgue measurable set]] <math>\Omega</math> in the [[Euclidean space]] <math>\R^n.</math> The functional
<math display=block>E : X \to \R</math>
<math display=block>E(u) = \int_\Omega F(u(x)) \, dx</math>
where <math>F</math> is a [[Real number|real]]-valued function of a real variable and <math>u</math> is defined on <math>\Omega</math> with real values, has Gateaux derivative
<math display=block>dE(u;\psi) = \langle F'(u),\psi \rangle := \int_\Omega F'(u(x))\,\psi(x) \,dx.</math>

Indeed, the above is the limit <math>\tau \to 0</math> of
<math display=block>\begin{align}
\frac{E(u+\tau\psi) - E(u)}{\tau} & = \frac{1}{\tau} \left( \int_\Omega F(u+\tau\,\psi)\,dx - \int_\Omega F(u)\,dx \right) \\[6pt]
& =\frac{1}{\tau} \left(\int_\Omega\int_0^1 \frac{d}{ds} F(u+s\,\tau\,\psi) \,ds\,dx\right) \\[6pt]
& =\int_\Omega\int_0^1 F'(u+s\tau\psi)\,\psi \,ds\,dx.
\end{align}</math>

==See also==

* {{annotated link|Hadamard derivative}}
* {{annotated link|Differentiable vector-valued functions from Euclidean space}}
* {{annotated link|Differentiation in Fréchet spaces}}
* {{annotated link|Fractal derivative}}
* {{annotated link|Generalizations of the derivative}}
* {{annotated link|Infinite-dimensional vector function}}
* {{annotated link|Quasi-derivative}}
* {{annotated link|Quaternionic analysis}}
* {{annotated link|Semi-differentiability}}

==References==

{{reflist}}
{{reflist|group=note}}
* {{Citation|last=Gateaux|first=René|title=Sur les fonctionnelles continues et les fonctionnelles analytiques|pages=325–327|url=http://gallica.bnf.fr/ark:/12148/bpt6k31103/f325.image|journal=Comptes rendus hebdomadaires des séances de l'Académie des sciences|publication-place=Paris|volume=157|year=1913|access-date=2 September 2012}}.
* {{Citation|last=Gateaux|first=René|title=Fonctions d'une infinité de variables indépendantes|journal=Bulletin de la Société Mathématique de France|volume=47|year=1919|pages=70–96|doi=10.24033/bsmf.995 |url=http://www.numdam.org:80/numdam-bin/item?id=BSMF_1919__47__70_1|doi-access=free}}.
* {{Citation|last=Hamilton|first=R. S.|author-link=Richard S. Hamilton|title=The inverse function theorem of Nash and Moser|url=http://projecteuclid.org/euclid.bams/1183549049|
journal=Bull. Amer. Math. Soc.|issue=1|year=1982|pages=65–222|doi=10.1090/S0273-0979-1982-15004-2|volume=7|mr=656198|doi-access=free}}
* {{Citation|last1=Hille|first1=Einar|last2=Phillips|first2=Ralph S.|author-link2=Ralph Phillips (mathematician)|title=Functional analysis and semi-groups|publisher=American Mathematical Society|location=Providence, R.I.|mr=0423094|year=1974}}.
* {{springer|last=Tikhomirov|first=V.M.|title=Gâteaux variation|id=G/g043390|year=2001}}.
* {{Citation|last1=Vainberg|first1=M.M.|title=Variational Methods for the Study of Nonlinear Operators|date=1964|publisher=Holden-Day, Inc|location=San Francisco, London, Amsterdam|page=57}}
* {{Citation|last1=Zorn|first1=Max|title=Characterization of analytic functions in Banach spaces|mr=0014190|jstor=1969198|year=1945|journal=[[Annals of Mathematics]]|series=Second Series|issn=0003-486X|volume=46|issue=4|doi=10.2307/1969198|pages=585–593}}.
* {{Citation|last=Zorn|first=Max|title=Derivatives and Frechet differentials|journal=Bulletin of the American Mathematical Society|year=1946|volume=52|pages=133–137|url=https://www.ams.org/bull/1946-52-02/S0002-9904-1946-08524-9/home.html|doi=10.1090/S0002-9904-1946-08524-9|issue=2|mr=0014595|doi-access=free}}.

{{Functional analysis}}

{{DEFAULTSORT:Gateaux Derivative}}
[[Category:Generalizations of the derivative]]
[[Category:Topological vector spaces]]

{{Sourceattribution|Gateaux derivative}}