View source for Inverse problem

{{Short description|Process of calculating the causal factors that produced a set of observations}}

An '''inverse problem''' in science is the process of calculating from a set of observations the causal factors that produced them: for example, calculating an image in [[Astronomy:X-ray computed tomography|X-ray computed tomography]], source reconstruction in acoustics, or calculating the density of the Earth from measurements of its gravity field. It is called an inverse problem because it starts with the effects and then calculates the causes. It is the inverse of a forward problem, which starts with the causes and then calculates the effects.

Inverse problems are some of the most important mathematical problems in [[Science|science]] and [[Mathematics|mathematics]] because they tell us about parameters that we cannot directly observe. They can be found in [[System identification|system identification]], [[Physics:Optics|optics]], [[Engineering:Radar|radar]], [[Physics:Acoustics|acoustics]], [[Social:Communication theory|communication theory]], [[Signal processing|signal processing]], [[Medicine:Medical imaging|medical imaging]], [[Computer vision|computer vision]],<ref>{{Cite book| last=Mohamad-Djafari|first=Ali| url=https://books.google.com/books?id=ef8DREm_9OMC&q=%22inverse+problem%22| title=Inverse Problems in Vision and 3D Tomography| date=2013-01-29| publisher=John Wiley & Sons|isbn=978-1-118-60046-7|language=en}}</ref><ref>Pizlo, Zygmunt. "[https://www.sciencedirect.com/science/article/pii/S0042698901001730 Perception viewed as an inverse problem]." Vision research 41.24 (2001): 3145-3161.</ref> [[Physics:Geophysics|geophysics]], [[Physics:Oceanography|oceanography]], [[Physics:Meteorology|meteorology]], [[Astronomy:Astronomy|astronomy]], [[Earth:Remote sensing|remote sensing]], [[Natural language processing|natural language processing]], [[Machine learning|machine learning]],<ref>Vito, Ernesto De, et al. "[http://www.jmlr.org/papers/volume6/devito05a/devito05a.pdf Learning from examples as an inverse problem]." Journal of Machine Learning Research 6.May (2005): 883-904.</ref> [[Engineering:Nondestructive testing|nondestructive testing]], slope stability analysis<ref>{{cite journal |last1= Cardenas |first1=IC|title= On the use of Bayesian networks as a meta-modeling approach to analyse uncertainties in slope stability analysis| journal =Georisk: Assessment and Management of Risk for Engineered Systems and Geohazards|date=2019 | volume=13|issue=1|pages=53–65 |doi=10.1080/17499518.2018.1498524 |bibcode=2019GAMRE..13...53C |s2cid=216590427}}</ref> and many other fields. 
==History==
Starting with the effects to discover the causes has concerned physicists for centuries. A historical example is the calculations of [[Biography:John Couch Adams|Adams]] and [[Biography:Urbain Le Verrier|Le Verrier]] which led to the discovery of [[Astronomy:Neptune|Neptune]] from the perturbed trajectory of [[Astronomy:Uranus|Uranus]]. However, a formal study of inverse problems was not initiated until the 20th century.

One of the earliest examples of a solution to an inverse problem was discovered by [[Biography:Hermann Weyl|Hermann Weyl]] and published in 1911, describing the asymptotic behavior of eigenvalues of the [[Laplace–Beltrami operator]].<ref>{{cite journal |last=Weyl |first=Hermann |url=http://gdz.sub.uni-goettingen.de/dms/load/img/?IDDOC=63048 |title=Über die asymptotische Verteilung der Eigenwerte |journal=Nachrichten der Königlichen Gesellschaft der Wissenschaften zu Göttingen |pages=110–117 |year=1911 |access-date=2018-05-14 |archive-url=https://web.archive.org/web/20130801090504/http://gdz.sub.uni-goettingen.de/dms/load/img/?IDDOC=63048 |archive-date=2013-08-01 }}</ref> Today known as ''Weyl's law'', it is perhaps most easily understood as an answer to the question of whether it is possible to [[Hearing the shape of a drum|hear the shape of a drum]]. Weyl conjectured that the eigenfrequencies of a drum would be related to the area and perimeter of the drum by a particular equation, a result improved upon by later mathematicians.

The field of inverse problems was later touched on by [[Place:Soviet Union|Soviet]]-Armenian physicist, Viktor Ambartsumian.<ref>[http://ambartsumian.ru/en/papers/epilogue-ambartsumian'-s-paper/ » Epilogue — Ambartsumian' s paper Viktor Ambartsumian<!-- Bot generated title -->]</ref><ref>{{cite journal|title=A life in astrophysics. Selected papers of Viktor A. Ambartsumian| first=Rouben V.| last=Ambartsumian| journal=Astrophysics| volume=41| issue=4| pages=328–330| doi=10.1007/BF02894658| year = 1998| s2cid=118952753}}</ref>

While still a student, Ambartsumian thoroughly studied the theory of atomic structure, the formation of energy levels, and the [[Schrödinger equation]] and its properties, and when he mastered the theory of [[Eigenvalues and eigenvectors|eigenvalues]] of [[Differential equation|differential equation]]s, he pointed out the apparent analogy between discrete energy levels and the eigenvalues of differential equations. He then asked: given a family of eigenvalues, is it possible to find the form of the equations whose eigenvalues they are? Essentially Ambartsumian was examining the inverse Sturm–Liouville problem, which dealt with determining the equations of a vibrating string. This paper was published in 1929 in the German physics journal ''[[Physics:Zeitschrift für Physik|Zeitschrift für Physik]]'' and remained in obscurity for a rather long time. Describing this situation after many decades, Ambartsumian said, "If an astronomer publishes an article with a mathematical content in a physics journal, then the most likely thing that will happen to it is oblivion."

Nonetheless, toward the end of the Second World War, this article, written by the 20-year-old Ambartsumian, was found by Swedish mathematicians and formed the starting point for a whole area of research on inverse problems, becoming the foundation of an entire discipline.

Then important efforts have been devoted to a "direct solution" of the inverse scattering problem especially by [[Physics:Marchenko equation|Gelfand and Levitan]] in the Soviet Union.<ref name="sciencedirect.com">{{cite journal |last1=Burridge |first1=Robert |title=The Gelfand-Levitan, the Marchenko, and the Gopinath-Sondhi integral equations of inverse scattering theory, regarded in the context of inverse impulse-response problems |journal=Wave Motion |date=1980 |volume=2 |issue=4 |pages=305–323 |doi=10.1016/0165-2125(80)90011-6 |bibcode=1980WaMot...2..305B }}</ref> They proposed an analytic constructive method for determining the solution.  When computers became available, some authors have investigated the possibility of applying their approach to similar problems such as the inverse problem in the 1D wave equation. But it rapidly turned out that the inversion is an unstable process: noise and errors can be tremendously amplified making a direct solution hardly practicable.
Then, around the seventies, the least-squares and probabilistic approaches came in and turned out to be very helpful for the determination of parameters involved in various physical systems. This approach met a lot of success. Nowadays inverse problems are also investigated in fields outside physics, such as chemistry, economics, and computer science. Eventually, as numerical models become prevalent in many parts of society, we may expect an inverse problem associated with each of these numerical models.

==Conceptual understanding==
Since Newton, scientists have extensively attempted to model the world. In particular, when a [[Mathematical model|mathematical model]] is available (for instance, Newton's gravitational law or Coulomb's equation for electrostatics), we can foresee, given some parameters that describe a physical system (such as a distribution of mass or a distribution of electric charges), the behavior of the system. This approach is known as mathematical modeling and the above-mentioned physical parameters are called the '''model parameters''' or simply the '''model'''. To be precise, we introduce the notion of '''state of the physical system''': it is the solution of the mathematical model's equation. In [[Optimal control|optimal control theory]], these equations are referred to as the [[State-space representation|state equations]]. In many situations we are not truly interested in knowing the physical state but just its effects on some objects (for instance, the effects the gravitational field has on a specific planet). Hence we have to introduce another operator, called the '''observation operator''', which converts the state of the physical system (here the predicted gravitational field) into what we want to observe (here the movements of the considered planet). We can now introduce the so-called '''forward problem''', which consists of two steps:
* determination of the state of the system from the physical parameters that describe it
* application of the observation operator to the estimated state of the system so as to predict the behavior of what we want to observe.
This leads to introduce another [[Operator (mathematics)|operator]] <math>F</math> (''F'' stands for "forward") which maps model parameters <math>p</math> into <math>F(p)</math>, the data that model <math>p</math> predicts that is the result of this two-step procedure. Operator <math>F</math> is called '''forward operator''' or '''forward map'''.
In this approach we basically attempt at predicting the effects knowing the causes.

The table below shows, the Earth being considered as the physical system and for different physical phenomena, the model parameters that describe the system, the physical quantity that describes the state of the physical system and observations commonly made on the state of the system. 
{| class="wikitable"
|-
! Governing equations !! Model parameters (input of the model)!! State of the physical system !! Common observations on the system
|-
| [[Physics:Newton's law of universal gravitation|Newton's law of gravity]] || Distribution of mass || [[Physics:Gravitational field|Gravitational field]] || Measurement made by [[Physics:Gravimeter|gravimeter]]s at different surface locations
|-
| [[Physics:Maxwell's equations|Maxwell's equations]] || Distribution of [[Physics:Magnetic susceptibility|magnetic susceptibility]] || [[Physics:Magnetic field|Magnetic field]] ||Magnetic field measured at different surface locations by [[Physics:Magnetometer|magnetometer]]s (case of a steady state)
|-
| [[Wave equation]] || Distribution of wave-speeds and densities || Wave-field caused by artificial or natural [[Earth:Seismic source|seismic source]]s || [[Physics:Particle velocity|Particle velocity]] measured by seismometers placed at different surface locations
|-
| [[Diffusion equation]] || Distribution of [[Physics:Mass diffusivity|Diffusion coefficient]] || Diffusing material concentration as a function of space and time || Monitoring of this concentration measured at different locations
|}
In the inverse problem approach we, roughly speaking, try to know the causes given the effects.

==General statement of the inverse problem==
The inverse problem is the "inverse" of the forward problem: instead of determining the data produced by particular model parameters, we want to determine the model parameters that produce the data <math>d_\text{obs}</math> that is the observation we have recorded (the subscript '''obs''' stands for observed).
Our goal, in other words, is to determine the model parameters <math>p</math> such that (at least approximately)
<math display="block"> d_\text{obs} = F(p)</math>
where <math>F</math> is the forward map. We denote by <math>M</math> the (possibly infinite) number of model parameters, and by <math>N</math> the number of recorded data.
We introduce some useful concepts and the associated notations that will be used below:
* The '''space of models''' denoted by <math>P</math>: the [[Vector space|vector space]] spanned by model parameters; it has <math>M</math> dimensions;
* The '''space of data''' denoted by <math>D</math>: <math>D = \R^N</math> if we organize the measured samples in a vector with <math>N</math> components (if our measurements consist of functions, <math>D</math> is a vector space with infinite dimensions);
* <math>F(p)</math>: the '''response of model''' <math>p</math>; it consists of the '''data predicted by model''' <math>p</math>;
* <math>F(P)</math>: the image of <math>P</math> by the forward map, it is a subset of <math>D</math> (but not a subspace unless <math>F</math> is linear) made of responses of all models;
* <math>d_\text{obs} - F(p)</math>: the '''data misfits (or residuals)''' associated with model  <math>p</math>: they can be arranged as a vector, an element of <math>D</math>.
The concept of residuals is very important: in the scope of finding a model that matches the data, '''their analysis reveals if the considered model can be considered as realistic or not'''. Systematic unrealistic discrepancies between the data and the model responses also reveals that the forward map is inadequate and may give insights about an improved forward map.

When operator <math>F</math> is linear, the inverse problem is linear. Otherwise, that is most often, the inverse problem is nonlinear.
Also, models cannot always be described by a finite number of parameters. It is the case when we look for [[Distributed parameter system|distributed parameters]] (a distribution of wave-speeds for instance): in such cases the goal of the inverse problem is to retrieve one or several functions. Such inverse problems are inverse problems with infinite dimension.

==<span id="Linear inverse problem"></span>Linear inverse problems==
In the case of a linear forward map and when we deal with a finite number of model parameters, the forward map can be written as a [[Linear system|linear system]]
<math display="block"> d = Fp</math>
where <math>F</math> is the [[Matrix (mathematics)|matrix]] that characterizes the forward map. The linear system can be systematically solved by means of both regularization and Bayesian methods.<ref>{{cite journal | last1 = Huang | first1 = Yunfei. | display-authors = etal | year = 2019 | title = Traction force microscopy with optimized regularization and automated Bayesian parameter selection for comparing cells | journal = Scientific Reports | volume = 9 | number = 1| page = 537 | article-number = 539 | doi = 10.1038/s41598-018-36896-x | pmid = 30679578 | doi-access = free | pmc = 6345967 | arxiv = 1810.05848 | bibcode = 2019NatSR...9..539H }}</ref> 

===An elementary example: Earth's gravitational field===
Only a few physical systems are actually linear with respect to the model parameters. One such system from geophysics is that of the [[Physics:Gravity of Earth|Earth's gravitational field]]. The Earth's gravitational field is determined by the density distribution of the Earth in the subsurface. Because the [[Earth:Lithology|lithology]] of the Earth changes quite significantly, we are able to observe minute differences in the Earth's gravitational field on the surface of the Earth. From our understanding of gravity (Newton's Law of Gravitation), we know that the mathematical expression for gravity is:
<math display="block">d= \frac{G p}{r^2};</math>
here <math>d</math> is a measure of the local gravitational acceleration, <math>G</math> is the [[Physics:Gravitational acceleration|universal gravitational constant]], <math>p</math> is the local mass (which is related to density) of the rock in the subsurface and <math>r</math> is the distance from the mass to the observation point.

By discretizing the above expression, we are able to relate the discrete data observations on the surface of the Earth to the discrete model parameters (density) in the subsurface that we wish to know more about. For example, consider the case where we have measurements carried out at 5 locations on the surface of the Earth. In this case, our data vector, <math>d</math> is a column vector of dimension (5×1): its <math>i</math>-th component is associated with the <math>i</math>-th observation location.  We also know that we only have five unknown masses <math>p_j</math> in the subsurface (unrealistic but used to demonstrate the concept) with known location: we denote by <math>r_{ij}</math> the distance between the <math>i</math>-th observation location and the <math>j</math>-th mass. Thus, we can construct the linear system relating the five unknown masses to the five data points as follows:
<math display="block">d = F p, </math>
<math display="block">d =
\begin{bmatrix} d_1 \\ d_2 \\ d_3 \\ d_4 \\ d_5 \end{bmatrix}, \quad
p = \begin{bmatrix} p_1 \\ p_2 \\ p_3 \\ p_4 \\ p_5 \end{bmatrix},</math>
<math display="block">F = \begin{bmatrix}
\frac{G}{r_{11}^2} & \frac{G}{r_{12}^2} & \frac{G}{r_{13}^2} & \frac{G}{r_{14}^2} & \frac{G}{r_{15}^2} \\
\frac{G}{r_{21}^2} & \frac{G}{r_{22}^2} & \frac{G}{r_{23}^2} & \frac{G}{r_{24}^2} & \frac{G}{r_{25}^2} \\
\frac{G}{r_{31}^2} & \frac{G}{r_{32}^2} & \frac{G}{r_{33}^2} & \frac{G}{r_{34}^2} & \frac{G}{r_{35}^2} \\
\frac{G}{r_{41}^2} & \frac{G}{r_{42}^2} & \frac{G}{r_{43}^2} & \frac{G}{r_{44}^2} & \frac{G}{r_{45}^2} \\
\frac{G}{r_{51}^2} & \frac{G}{r_{52}^2} & \frac{G}{r_{53}^2} & \frac{G}{r_{54}^2} & \frac{G}{r_{55}^2}
\end{bmatrix}</math>

To solve for the model parameters that fit our data, we might be able to invert the matrix <math>F</math> to directly convert the measurements into our model parameters. For example:
<math display="block">p = F^{-1} d_\text{obs} </math>
A system with five equations and five unknowns is a very specific situation: our example was designed to end up with this specificity. In general, the numbers of data and unknowns are different so that matrix <math>F</math> is not square.

However, even a square matrix can have no inverse: matrix <math>F</math> can be [[Rank (linear algebra)|rank]] deficient (i.e. has zero eigenvalues) and the solution of the system <math>p = F^{-1} d_\text{obs} </math> is not unique. Then the solution of the inverse problem will be undetermined. This is a first difficulty. Over-determined systems (more equations than unknowns) have other issues.
Also noise may corrupt our observations making <math>d</math> possibly outside the space <math>F(P)</math> of possible responses to model parameters so that solution of the system <math>p = F^{-1} d_\text{obs} </math> may not exist. This is another difficulty.

==== Tools to overcome the first difficulty ====
The first difficulty reflects a crucial problem: Our observations do not contain enough information and additional data are required. Additional data can come from physical '''prior information''' on the parameter values, on their spatial distribution or, more generally, on their mutual dependence. It can also come from other experiments: For instance, we may think of integrating data recorded by gravimeters and seismographs for a better estimation of densities. 
The integration of this additional information is basically a problem of [[Statistics|statistics]]. This discipline is the one that can answer the question: How to mix quantities of different nature? We will be more precise in the section "Bayesian approach" below.

Concerning distributed parameters, prior information about their spatial distribution often consists of information about some derivatives of these distributed parameters. Also, it is common practice, although somewhat artificial, to look for the "simplest" model that reasonably matches the data. This is usually achieved by [[Penalty method|penalizing]] the [[Lp space|<math>L^1</math> norm]] of the gradient (or the [[Total variation|total variation]]) of the parameters (this approach is also referred to as the maximization of the entropy). One can also make the model simple through a parametrization that introduces degrees of freedom only when necessary.

Additional information may also be integrated through inequality constraints on the model parameters or some functions of them. Such constraints are important to avoid unrealistic values for the parameters (negative values for instance). In this case, the space spanned by model parameters will no longer be a vector space but a '''subset of admissible models''' denoted by <math>P_\text{adm}</math> in the sequel.

==== Tools to overcome the second difficulty ====
As mentioned above, noise may be such that our measurements are not the image of any model, so that we cannot look for a model that produces the data but rather look for [[Model selection|the best (or optimal) model]]: that is, the one that best matches the data. This leads us to minimize an objective function, namely a [[Functional (mathematics)|functional]]  that quantifies how big the residuals are or how far the predicted data are from the observed data. Of course, when we have perfect data (i.e. no noise) then the recovered model should fit the observed data perfectly. A standard objective function, <math>\varphi</math>, is of the form:
<math>\varphi(p) = \|F p-d_\text{obs} \|^2 </math>
where <math>\| \cdot \| </math> is the Euclidean norm (it will be the [[Lp space|<math>L^2</math> norm]] when the measurements are functions instead of samples) of the residuals. This approach amounts to making use of [[Least squares|ordinary least squares]], an approach widely used in statistics. However, the Euclidean norm is known to be very sensitive to outliers: to avoid this difficulty we may think of using other distances, for instance the <math>L^1</math> norm, in replacement of the <math>L^2</math> norm.

==== Bayesian approach ====
Very similar to the least-squares approach is the probabilistic approach: If we know the statistics of the noise that contaminates the data, we can think of seeking the most likely model m, which is the model that matches the [[Maximum likelihood estimation|maximum likelihood criterion]]. If the noise is [[Normal distribution|Gaussian]], the maximum likelihood criterion appears as a least-squares criterion, the Euclidean scalar product in data space being replaced by a scalar product involving the [[Covariance|co-variance]] of the noise. Also, should prior information on model parameters be available, we could think of using [[Bayesian inference]] to formulate the solution of the inverse problem. This approach is described in detail in Tarantola's book.<ref>{{cite book |last1=Tarantola |first1=Albert |title=Inverse problem theory |url=https://archive.org/details/inverseproblemth0000tara |url-access=registration |date=1987 |publisher=Elsevier |isbn=978-0-444-59967-4 |edition=1st}}</ref>

==== Numerical solution of our elementary example ====
Here we make use of the Euclidean norm to quantify the data misfits. As we deal with a linear inverse problem, the objective function is quadratic. For its minimization, it is classical to compute its gradient using the same rationale (as we would to minimize a function of only one variable). At the optimal model <math>p_\text{opt}</math>, this gradient vanishes which can be written as:
<math display="block">\nabla_p \varphi = 2 (F^\mathrm{T} F p_\text{opt} - F^\mathrm{T} d_\text{obs}) = 0 </math>
where ''F''<sup>T</sup> denotes the matrix transpose of ''F''. This equation simplifies to:
<math display="block">F^\mathrm{T} F p_\text{opt} = F^\mathrm{T} d_\text{obs} </math>

This expression is known as the [https://en.wikipedia.org/?title=Normal_equations&redirect=no normal equation] and gives us a possible solution to the inverse problem. 
In our example matrix <math>F^\mathrm{T} F</math> turns out to be generally full rank so that the equation above makes sense and determines uniquely the model parameters: we do not need integrating additional information for ending up with a unique solution.

=== Mathematical and computational aspects ===

Inverse problems are typically ill-posed, as opposed to the [[Well-posed problem|well-posed problem]]s usually met in mathematical modeling. Of the three conditions for a [[Well-posed problem|well-posed problem]] suggested by [[Biography:Jacques Hadamard|Jacques Hadamard]] (existence, uniqueness, and stability of the solution or solutions) the condition of stability is most often violated. In the sense of [[Functional analysis|functional analysis]], the inverse problem is represented by a mapping between [[Metric space|metric space]]s. While inverse problems are often formulated in infinite dimensional spaces, limitations to a finite number of measurements, and the practical consideration of recovering only a finite number of unknown parameters, may lead to the problems being recast in discrete form. In this case the inverse problem will typically be ''[[Condition number|ill-conditioned]]''. In these cases, [[Regularization (mathematics)|regularization]] may be used to introduce mild assumptions on the solution and prevent [[Overfitting|overfitting]]. Many instances of regularized inverse problems can be interpreted as special cases of [[Bayesian inference]].<ref>{{cite book| chapter-url=http://www.ipgp.fr/~tarantola/Files/Professional/Books/InverseProblemTheory.pdf| title=Inverse Problem Theory and Methods for Model Parameter Estimation| pages=i–xii| first=Albert|last=Tarantola| publisher=SIAM| doi=10.1137/1.9780898717921.fm| chapter=Front Matter| year=2005| isbn=978-0-89871-572-9}}</ref>

==== Numerical solution of the optimization problem ====
Some inverse problems have a very simple solution, for instance, when one has a set of [[Unisolvent functions|unisolvent functions]], meaning a set of {{tmath|n}} functions such that evaluating them at {{tmath|n}} distinct points yields a set of linearly independent vectors. This means that given a [[Linear combination|linear combination]] of these functions, the coefficients can be computed by arranging the vectors as the columns of a matrix and then inverting this matrix. The simplest example of unisolvent functions is polynomials constructed, using the unisolvence theorem, so as to be unisolvent. Concretely, this is done by inverting the [[Vandermonde matrix]]. But this a very specific situation.

In general, the solution of an inverse problem requires sophisticated optimization algorithms. When the model is described by a large number of parameters (the number of unknowns involved in some diffraction tomography applications can reach one billion), solving the linear system associated with the normal equations can be cumbersome. The numerical method to be used for solving the optimization problem depends in particular on the cost required for computing the solution  <math>F p</math> of the forward problem. Once chosen the appropriate algorithm for solving the forward problem (a straightforward matrix-vector multiplication may be not adequate when matrix <math>F</math> is huge), the appropriate algorithm for carrying out the minimization can be found in textbooks dealing with numerical methods for the solution of linear systems and for the minimization of quadratic functions (see for instance Ciarlet<ref>{{cite book |last1=Ciarlet |first1=Philippe |title=Introduction à l'analyse numérique matricielle et à l'optimisation |date=1994 |publisher=Masson |location=Paris |isbn=978-2-225-68893-5}}</ref> or Nocedal<ref>{{cite book |last1=Nocedal |first1=Jorge |title=Numerical optimization |date=2006 |publisher=Springer}}</ref>).

Also, the user may wish to add physical constraints to the models: In this case, they have to be familiar with [[Constrained optimization|constrained optimization methods]], a subject in itself. In all cases, computing the gradient of the objective function often is a key element for the solution of the optimization problem. As mentioned above, information about the spatial distribution of a distributed parameter can be introduced through the parametrization. One can also think of adapting this parametrization during the optimization.<ref>{{cite journal |last1=Ben Ameur |first1=Hend |last2=Chavent |first2=Guy |last3=Jaffré |first3=Jérôme |title=Refinement and coarsening indicators for adaptive parametrization: application to the estimation of hydraulic transmissivities |journal=Inverse Problems |date=2002 |volume=18 |issue=3 |pages=775–794 |url=https://hal.inria.fr/docs/00/07/22/95/PDF/RR-4292.pdf|doi=10.1088/0266-5611/18/3/317 |bibcode=2002InvPr..18..775B |s2cid=250892174 }}</ref>

Should the objective function be based on a norm other than the Euclidean norm, we have to leave the area of quadratic optimization. As a result, the optimization problem becomes more difficult. In particular, when the <math>L^1</math> norm is used for quantifying the data misfit the objective function is no longer differentiable: its gradient does not make sense any longer. Dedicated methods (see for instance Lemaréchal<ref>{{cite book |last1=Lemaréchal |first1=Claude |title=Optimization, Handbooks in Operations Research and Management Science |date=1989 |publisher=Elsevier |pages=529–572}}</ref>) from non differentiable optimization come in.

Once the optimal model is computed we have to address the question: "Can we trust this model?" The question can be formulated as follows: How large is the set of models that match the data "nearly as well" as this model? In the case of quadratic objective functions, this set is contained in a hyper-ellipsoid, a subset of <math>R^M</math> (<math>M</math> is the number of unknowns), whose size depends on what we mean with "nearly as well", that is on the noise level. The direction of the largest axis of this ellipsoid ([[Eigenvalues and eigenvectors|eigenvector]] associated with the smallest eigenvalue of matrix <math>F^T F</math>) is the direction of poorly determined components: if we follow this direction, we can bring a strong perturbation to the model without changing significantly the value of the objective function and thus end up with a significantly different quasi-optimal model. We clearly see that the answer to the question "can we trust this model" is governed by the noise level and by the eigenvalues of the [[Hessian matrix|Hessian]] of the objective function or equivalently, in the case where no regularization has been integrated, by the [[Singular value|singular value]]s of matrix <math>F</math>. Of course, the use of regularization (or other kinds of prior information) reduces the size of the set of almost optimal solutions and, in turn, increases the confidence we can put in the computed solution.

==== Stability, regularization and model discretization in infinite dimension ====

We focus here on the recovery of a distributed parameter.
When looking for distributed parameters we have to discretize these unknown functions. Doing so, we reduce the dimension of the problem to something finite. But now, the question is: is there any link between the solution we compute and that of the initial problem? Then another question: what do we mean with the solution of the initial problem? Since a finite number of data does not allow the determination of an infinity of unknowns, the original data misfit functional has to be regularized to ensure the uniqueness of the solution. Many times, reducing the unknowns to a finite-dimensional space will provide an adequate regularization: the computed solution will look like a discrete version of the solution we were looking for. For example, a naïve discretization will often work for solving the [[Deconvolution|deconvolution]] problem: it will work as long as we do not allow missing frequencies to show up in the numerical solution. But many times, regularization has to be integrated explicitly in the objective function.

In order to understand what may happen, we have to keep in mind that solving such a linear inverse problem amount to solving a Fredholm integral equation of the first kind: 
<math display="block">d(x) = \int_\Omega K(x,y) p(y) dy</math>

where <math>K</math> is the kernel, <math>x</math> and <math>y</math> are vectors of <math>R^2</math>, and <math>\Omega</math> is a domain in <math>R^2</math>. This holds for a 2D application. For a 3D application, we consider <math> x,y \in R^3</math>. Note that here the model parameters <math>p</math> consist of a function and that the response of a model also consists of a function denoted by <math>d(x)</math>. This equation is an extension to infinite dimension of the matrix equation <math>d=Fp</math> given in the case of discrete problems.

For sufficiently smooth <math>K</math> the operator defined above is [[Compact operator|compact]] on reasonable [[Banach space]]s such as the [[Lp space|<math>L^2</math>]]. [[Compact operator|F. Riesz theory]] states that the set of singular values of such an operator contains zero (hence the existence of a null-space), is finite or at most countable, and, in the latter case, they constitute a sequence that goes to zero. In the case of a symmetric kernel, we have an infinity of eigenvalues and the associated eigenvectors constitute a hilbertian basis of <math>L^2</math>.  Thus any solution of this equation is determined up to an additive function in the null-space and, in the case of infinity of singular values, the solution (which involves the reciprocal of arbitrary small eigenvalues) is unstable: two ingredients that make the solution of this integral equation a typical ill-posed problem! However, we can define a solution through the [[Generalized inverse|pseudo-inverse]] of the forward map (again up to an arbitrary additive function). When the forward map is compact, the classical [[Tikhonov regularization]] will work if we use it for integrating prior information stating that the <math>L^2</math> norm of the solution should be as small as possible: this will make the inverse problem well-posed. Yet, as in the finite dimension case, we have to question the confidence we can put in the computed solution. Again, basically, the information lies in the eigenvalues of the Hessian operator. Should subspaces containing eigenvectors associated with small eigenvalues be explored for computing the solution, then the solution can hardly be trusted: some of its components will be poorly determined. The smallest eigenvalue is equal to the weight introduced in Tikhonov regularization.

Irregular kernels may yield a forward map which is not compact and even [[Unbounded operator|unbounded]] if we naively equip the space of models with the <math>L^2</math> norm. In such cases, the Hessian is not a bounded operator and the notion of eigenvalue does not make sense any longer. A mathematical analysis is required to make it a [[Bounded operator|bounded operator]] and design a well-posed problem: an illustration can be found in.<ref>{{cite journal |last1=Delprat-Jannaud |first1=Florence |last2=Lailly |first2=Patrick |title=Ill-posed and well-posed formulations of the reflection travel time tomography problem|journal=Journal of Geophysical Research |date=1993 |volume=98 |issue=B4 |pages=6589–6605 |doi=10.1029/92JB02441 |bibcode=1993JGR....98.6589D }}</ref> Again, we have to question the confidence we can put in the computed solution and we have to generalize the notion of eigenvalue to get the answer.<ref>{{cite journal |last1=Delprat-Jannaud |first1=Florence |last2=Lailly |first2=Patrick |title=What information on the Earth model do reflection traveltimes provide |journal=Journal of Geophysical Research |date=1992 |volume=98 |issue=B13 |pages=827–844|doi=10.1029/92JB01739 |bibcode=1992JGR....9719827D }}</ref>

Analysis of the spectrum of the Hessian operator is thus a key element to determine how reliable the computed solution is.  However, such an analysis is usually a very heavy task. This has led several authors to investigate alternative approaches in the case where we are not interested in all the components of the unknown function but only in sub-unknowns that are the images of the unknown function by a linear operator. These approaches are referred to as the " Backus and Gilbert method<ref>{{cite journal |last1=Backus |first1=George |last2=Gilbert |first2=Freeman |title=The Resolving Power of Gross Earth Data |journal=Geophysical Journal of the Royal Astronomical Society |date=1968 |volume=16 |issue=10 |pages=169–205 |doi=10.1111/j.1365-246X.1968.tb00216.x |bibcode=1968GeoJ...16..169B |doi-access=free }}</ref>", [[Biography:Jacques-Louis Lions|Lions]]'s sentinels approach,<ref>{{cite journal |last1=Lions |first1=Jacques Louis |title=Sur les sentinelles des systèmes distribués |journal=C. R. Acad. Sci. Paris |date=1988 |series=I Math |pages=819–823}}</ref> and the SOLA method:<ref>{{cite journal |last1=Pijpers |first1=Frank |last2=Thompson |first2=Michael |title=The SOLA method for helioseismic inversion |journal=Astronomy and Astrophysics |date=1993 |volume=281 |issue=12 |pages=231–240|bibcode=1994A&A...281..231P }}</ref> these approaches turned out to be strongly related with one another as explained in Chavent<ref>{{cite book |last1=Chavent |first1=Guy |title=Least-Squares, Sentinels and Substractive Optimally Localized Average in Equations aux dérivées partielles et applications |date=1998 |publisher=Gauthier Villars |location=Paris |pages=345–356 |url=https://hal.inria.fr/inria-00073357/document}}</ref>   Finally, the concept of [[Physics:Optical resolution|limited resolution]], often invoked by physicists, is nothing but a specific view of the fact that some poorly determined components may corrupt the solution. But, generally speaking, these poorly determined components of the model are not necessarily associated with high frequencies.

===Some classical linear inverse problems for the recovery of distributed parameters===
The problems mentioned below correspond to different versions of the Fredholm integral: each of these is associated with a specific kernel <math>K</math>.

====Deconvolution====
The goal of [[Deconvolution|deconvolution]] is to reconstruct the original image or signal <math>p(x)</math> which appears as noisy and blurred on the data <math>d(x)</math>.<ref>Kaipio, J., & Somersalo, E. (2010). Statistical and computational inverse problems. New York, NY: Springer.</ref>
From a mathematical point of view, the kernel <math>K(x,y)</math> here only depends on the difference between <math>x</math> and <math>y</math>.

====Tomographic methods====
In these methods we attempt at recovering a distributed parameter, the observation consisting in the measurement of the integrals of this parameter carried out along a family of lines. We denote by <math>\Gamma_x</math> the line in this family associated with measurement point <math>x</math>. The observation at <math>x</math> can thus be written as:
<math display="block">d(x) = \int_{\Gamma_x} w(x,y) p(y) \, dy</math>
where <math>w(x,y)</math> a known weighting function. Comparing this equation with the Fredholm integral above, we notice that the kernel <math>K(x,y)</math> is kind of a [[Dirac delta function|delta function]] that peaks on line <math>{\Gamma_x}</math>. With such a kernel, the forward map is not compact.

===== Computed tomography =====
In [[Astronomy:X-ray computed tomography|X-ray computed tomography]] the lines on which the parameter is integrated are straight lines: the [[Tomographic reconstruction|tomographic reconstruction]] of the parameter distribution is based on the inversion of the [[Radon transform]]. Although from a theoretical point of view many linear inverse problems are well understood, problems involving the Radon transform and its generalisations still present many practical challenges, such as with of sufficiency of data. Such problems include incomplete data for the x-ray transform in three dimensions and problems involving the generalisation of the x-ray transform to tensor fields. Solutions explored include Algebraic Reconstruction Technique, filtered backprojection, and as computing power has increased, [[Iterative reconstruction|iterative reconstruction]] methods such as [[Physics:SAMV (algorithm)|iterative Sparse Asymptotic Minimum Variance]].<ref name=AbeidaZhang>{{cite journal | last1=Abeida | first1=Habti | last2=Zhang | first2=Qilin | last3=Li | first3=Jian | last4=Merabtine | first4=Nadjim | title=Iterative Sparse Asymptotic Minimum Variance Based Approaches for Array Processing | journal=IEEE Transactions on Signal Processing | volume=61 | issue=4 | year=2013 | issn=1053-587X | doi=10.1109/tsp.2012.2231676 | pages=933–944 | url=https://qilin-zhang.github.io/_pages/pdfs/SAMVpaper.pdf | arxiv=1802.03070 | bibcode=2013ITSP...61..933A | s2cid=16276001 }}</ref>

===== Diffraction tomography =====
Diffraction tomography is a classical linear inverse problem in exploration seismology: the amplitude recorded at one time for a given source-receiver pair is the sum of contributions arising from points such that the sum of the distances, measured in traveltimes, from the source and the receiver, respectively, is equal to the corresponding recording time. In 3D the parameter is not integrated along lines but over surfaces. Should the propagation velocity be constant, such points are distributed on an ellipsoid. The inverse problems consists in retrieving the distribution of diffracting points from the seismograms recorded along the survey, the velocity distribution being known.  A direct solution has been originally proposed by [http://amath.colorado.edu/~beylkin/papers/BEYLKI-1983a.pdf Beylkin] and Lambaré et al.:<ref>{{cite journal |last1=Lambaré |first1=Gilles |last2=Virieux |first2=Jean |last3=Madariaga |first3=Raul |last4=Jin |first4=Side |title=Iterative asymptotic inversion in the acoustic approximation |journal=  Geophysics |date=1992 |volume=57 |issue=9 |pages=1138–1154 |doi=10.1190/1.1443328 |bibcode=1992Geop...57.1138L |s2cid=55836067 }}</ref> these works were the starting points of approaches known as amplitude preserved migration (see Beylkin<ref>{{cite journal |last1=Beylkin |first1=Gregory |title=The inversion problem and applications of The generalized Radon transform |journal=Communications on Pure and Applied Mathematics |date=1984 |volume=XXXVII |issue=5 |pages=579–599 |url=http://amath.colorado.edu/faculty/beylkin/papers/BEYLKI-1984.pdf |doi=10.1002/cpa.3160370503 |bibcode=1984CPAM...37..579B }}</ref><ref>{{cite journal |last1=Beylkin |first1=Gregory |title=Imaging of discontinuities in the inverse scaterring problem by inversion of a causal generalized Radon transform |journal=J. Math. Phys. |date=1985 |volume=26 |issue=1 |pages=99–108|doi=10.1063/1.526755 |bibcode=1985JMP....26...99B }}</ref> and Bleistein<ref>{{cite journal |last1=Bleistein |first1=Norman |title=On the imaging of reflectors in the earth |journal=Geophysics |date=1987 |volume=52 |issue=7 |pages=931–942 |doi=10.1190/1.1442363 |bibcode=1987Geop...52..931B |s2cid=5095133 }}</ref>).  Should geometrical optics techniques (i.e. [https://www.encyclopediaofmath.org/index.php/Ray_method rays])  be used for the solving the wave equation, these methods turn out to be closely related to the so-called least-squares migration methods<ref>{{cite journal |last1=Nemeth |first1=Tamas |last2=Wu |first2=Chengjun |last3=Schuster |first3=Gerard |title=Least-squares migration of incomplete reflection data |journal = Geophysics |date=1999 |volume=64 |issue=1 | pages=208–221 | url=https://csim.kaust.edu.sa/web/FWI&LSM_papers/LSM/Geophysics1999Nemeth.pdf |doi=10.1190/1.1444517 |bibcode=1999Geop...64..208N }}</ref> derived from the least-squares approach (see Lailly,<ref name='Proceedings of the international conference on "Inverse Scattering, theory and applications", Tulsa, Oklahoma'>{{cite book |last1=Lailly |first1=Patrick |title=The seismic inverse problem as a sequence of before stack migrations |date=1983 |publisher=SIAM |location=Philadelphia |isbn=0-89871-190-8 |pages=206–220}}</ref> Tarantola<ref>{{Cite journal | doi=10.1190/1.1441754| title=Inversion of seismic reflection data in the acoustic approximation| journal=Geophysics| volume=49| issue=8| pages=1259–1266| year=1984| last1=Tarantola| first1=Albert | bibcode=1984Geop...49.1259T| s2cid=7596552}}</ref>).

===== {{anchor|Doppler tomography}}Doppler tomography (astrophysics) =====

If we consider a rotating stellar object, the spectral lines we can observe on a spectral profile will be shifted due to Doppler effect. Doppler tomography aims at converting the information contained in spectral monitoring of the object into a 2D image of the emission (as a function of the radial velocity and of the phase in the periodic rotation movement) of the stellar atmosphere. As explained by Tom Marsh<ref>{{cite journal |last1=Marsh |first1=Tom |title=Doppler tomography |journal=Astrophysics and Space Science |volume=296 |date=2005 |issue=1–4 |pages=403–415 |doi=10.1007/s10509-005-4859-3 |arxiv=astro-ph/0011020 |bibcode=2005Ap&SS.296..403M |s2cid=15334110 }}</ref> this linear inverse problem is tomography like: we have to recover a distributed parameter which has been integrated along lines to produce its effects in the recordings.

==== Inverse heat conduction ====

Early publications on inverse heat conduction arose from determining surface heat flux during atmospheric re-entry from buried temperature sensors.<ref>{{Cite journal | author1 = Shumakov, N. V. | title = A method for the experimental study of the process of heating a solid body | journal = Soviet Physics –Technical Physics (Translated by American Institute of Physics) | volume = 2 | page = 771 | year =  1957 }}</ref><ref>{{Cite journal | author1 = Stolz, G. Jr. | title =  Numerical solutions to an inverse problem of heat conduction for simple shapes | journal = Journal of Heat Transfer | volume =  82 | pages = 20–26 | year = 1960 | issue = 1 | doi = 10.1115/1.3679871 | bibcode = 1960ATJHT..82...20S }}</ref>
Other applications where surface heat flux is needed but surface sensors are not practical include: inside reciprocating engines, inside rocket engines; and, testing of nuclear reactor components.<ref>{{Cite book | author1 = Beck, J. V. | author2 =  Blackwell, B.  |author3 =  St. Clair, C. R. Jr. | title = Inverse Heat Conduction. Ill-Posed Problems | location =  New York | publisher = J. Wiley & Sons | year = 1985 | isbn =  0-471-08319-4 }}</ref> A variety of numerical techniques have been developed to address the ill-posedness and sensitivity to measurement error caused by damping and lagging in the temperature signal.<ref>{{cite journal | author1 = Beck, J. V. | author2 =  Blackwell, B. |author3 =  Haji-Sheikh, B. | title =  Comparison of some inverse heat conduction methods using experimental data | journal =  International Journal of Heat and Mass Transfer | volume = 39 | issue = 17 | year =  1996  | pages = 3649–3657 | doi =  10.1016/0017-9310(96)00034-8
| bibcode = 1996IJHMT..39.3649B }}</ref><ref>{{cite book | author1 = Ozisik, M. N. |author2 =  Orlande, H. R. B.
| title =  Inverse Heat Transfer, Fundamentals and Applications | publisher = CRC Press | year = 2021 | isbn = 978-0-367-82067-1 | edition = 2nd }}</ref><ref>{{cite book | title = Inverse Engineering Handbook, edited by K. A. Woodbury | publisher = CRC Press | year = 2002 | isbn = 978-0-8493-0861-1 }}</ref>

==Non-linear inverse problems==

Non-linear inverse problems constitute an inherently more difficult family of inverse problems. Here the forward map <math>F</math> is a non-linear operator. Modeling of physical phenomena often relies on the solution of a partial differential equation (see table above except for gravity law): although these partial differential equations are often linear, the physical parameters that appear in these equations depend in a non-linear way of the state of the system and therefore on the observations we make on it.

=== Some classical non-linear inverse problems ===

==== Inverse scattering problems ====

Whereas linear inverse problems were completely solved from the theoretical point of view at the end of the nineteenth century , only one class of nonlinear inverse problems was so before 1970, that of inverse spectral and (one space dimension) [[Physics:Inverse scattering problem|inverse scattering problem]]s, after the seminal work of the Russian mathematical school (Krein, [[Biography:Israel Gelfand|Gelfand]], Levitan, [[Biography:Vladimir Marchenko|Marchenko]]). A large review of the results has been given by Chadan and Sabatier in their book "Inverse Problems of Quantum Scattering Theory" (two editions in English, one in Russian).

In this kind of problem, data are properties of the spectrum of a linear operator which describe the scattering. The spectrum is made of eigenvalues and [[Eigenfunction|eigenfunction]]s, forming together the "discrete spectrum", and generalizations, called the continuous spectrum. The very remarkable physical point is that scattering experiments give information only on the continuous spectrum, and that knowing its full spectrum is both necessary and sufficient in recovering the scattering operator. Hence we have invisible parameters, much more interesting than the null space which has a similar property in linear inverse problems. In addition, there are physical motions in which the spectrum of such an operator is conserved as a consequence of such motion. This phenomenon is governed by special nonlinear partial differential evolution equations, for example the [[Korteweg–de Vries equation]]. If the spectrum of the operator is reduced to one single eigenvalue, its corresponding motion is that of a single bump that propagates at constant velocity and without deformation, a solitary wave called a "[[Soliton|soliton]]".

A perfect signal and its generalizations for the Korteweg–de Vries equation or other integrable nonlinear partial differential equations are of great interest, with many possible applications. This area has been studied as a branch of mathematical physics since the 1970s. Nonlinear inverse problems are also currently studied in many fields of applied science (acoustics, mechanics, quantum mechanics, electromagnetic scattering - in particular radar soundings, seismic soundings, and nearly all imaging modalities).

A final example related to the [[Riemann hypothesis]] was given by Wu and Sprung, the idea is that in the [[Physics:Semiclassical physics|semiclassical]] [[Physics:Old quantum theory|old quantum theory]] the inverse of the potential inside the Hamiltonian is proportional to the half-derivative of the eigenvalues (energies) counting function&nbsp;''n''(''x'').

====Permeability matching in oil and gas reservoirs====
The goal is to recover the diffusion coefficient in the [[Diffusion equation|parabolic partial differential equation]] that models single phase fluid flows in porous media. This problem has been the object of many studies since a pioneering work carried out in the early seventies.<ref>{{cite journal |last1=Chavent |first1=Guy |last2=Lemonnier |first2=Patrick |last3=Dupuy |first3=Michel |title=History Matching by Use of Optimal Control Theory |journal=Society of Petroleum Engineers Journal |date=1975 |volume=15 |issue=2 |pages=74–86 |doi=10.2118/4627-PA}}</ref> Concerning two-phase flows an important problem is to estimate the relative permeabilities and the capillary pressures.<ref>{{cite journal |last1=Chavent |first1=Guy |last2=Cohen |first2=Gary |last3=Espy |first3=M. |title=Determination of relative permeabilities and capillary pressures by an automatic adjustment method |journal=Society of Petroleum Engineers |date=1980 |issue=January |article-number=SPE-9237-MS |doi=10.2118/9237-MS}}</ref>

==== Inverse problems in the wave equations ====

The goal is to recover the wave-speeds (P and S waves) and the density distributions from [[Earth:Seismogram|seismogram]]s. Such inverse problems are of prime interest in seismology and [[Earth:Exploration geophysics|exploration geophysics]].
We can basically consider two mathematical models:
* The [[Wave equation|acoustic wave equation]] (in which S waves are ignored when the space dimensions are 2 or 3)
* The [[Physics:Linear elasticity|elastodynamics equation]] in which the P and S wave velocities can be derived from the [[Physics:Lamé parameters|Lamé parameters]] and the density.
These basic [[Hyperbolic partial differential equation|hyperbolic equations]] can be upgraded by incorporating [[Physics:Attenuation|attenuation]], [[Anisotropy|anisotropy]], and other physical effects.

The solution of the inverse problem in the 1D wave equation has been the object of many studies. It is one of the very few non-linear inverse problems for which we can prove the uniqueness of the solution.<ref name="sciencedirect.com"/> The analysis of the stability of the solution was another challenge.<ref name="ReferenceA">{{cite journal |last1=Bamberger |first1=Alain |last2=Chavent |first2=Guy |last3=Lailly |first3=Patrick |title=About the stability of the inverse problem in the 1D wave equation, application to the interpretation of the seismic profiles |journal=Journal of Applied Mathematics and Optimization |date=1979 |volume=5 |pages=1–47 |doi=10.1007/bf01442542 |s2cid=122428594 }}</ref> Practical applications, using the least-squares approach, were developed.<ref name="ReferenceA"/><ref>{{cite journal |last1=Macé |first1=Danièle |last2=Lailly |first2=Patrick |title=Solution of the VSP one dimensional inverse problem |journal=Geophysical Prospecting |date=1986 |volume=34 |issue=7 |pages=1002–1021 |osti=6901651 |doi=10.1111/j.1365-2478.1986.tb00510.x |bibcode=1986GeopP..34.1002M }}</ref>
Extension to 2D or 3D problems and to the elastodynamics equations was attempted since the 80's but turned out to be very difficult ! This problem often referred to as Full Waveform Inversion (FWI), is not yet completely solved: among the main difficulties are the existence of non-Gaussian noise into the seismograms, cycle-skipping issues (also known as phase ambiguity), and the chaotic behavior of the data misfit function.<ref>{{cite journal |last1=Virieux |first1=Jean |last2=Operto |first2=Stéphane |title=An overview of full-waveform inversion in exploration geophysics |journal= Geophysics|date=2009 |volume=74 |issue=6 |pages=WCC1–WCC26 |url=https://www.researchgate.net/publication/228078264 |doi=10.1190/1.3238367}}</ref> Some authors have investigated the possibility of reformulating the inverse problem so as to make the objective function less chaotic than the data misfit function.<ref name="ReferenceB">{{cite journal |last1=Clément |first1=François |last2=Chavent |first2=Guy |last3=Gomez |first3=Suzana |title=Migration-based traveltime waveform inversion of 2-D simple structures: A synthetic example |journal= Geophysics |date=2001 |volume=66 |issue=3 |pages=845–860|doi=10.1190/1.1444974 |bibcode=2001Geop...66..845C }}</ref><ref name="ReferenceC">{{cite journal |last1=Symes |first1=William |last2=Carrazone |first2=Jim |title=Velocity inversion by Differential semblance optimization |journal= Geophysics |date=1991 |volume=56 |issue=5 |pages=654–663 |doi=10.1190/1.1443082 |bibcode=1991Geop...56..654S }}</ref>

==== Travel-time tomography ====

Realizing how difficult is the inverse problem in the wave equation, seismologists investigated a simplified approach making use of geometrical optics. In particular they aimed at inverting for the propagation velocity distribution, knowing the arrival times of wave-fronts observed on seismograms. These wave-fronts can be associated with direct arrivals or with reflections associated with reflectors whose geometry is to be determined, jointly with the velocity distribution.

The arrival time distribution <math>{\tau}(x)</math> (<math>x</math> is a point in physical space) of a wave-front issued from a point source, satisfies the [[Eikonal equation]]:
<math display="block">\|\nabla \tau (x)\| =  s(x),</math>
where <math>s(x)</math> denotes the [[Physics:Slowness (seismology)|slowness]] (reciprocal of the velocity) distribution. The presence of <math>\| \cdot \| </math> makes this equation nonlinear. It is classically solved by shooting [[Physics:Ray tracing|rays]] (trajectories about which the arrival time is stationary) from the point source.

This problem is tomography like: the measured arrival times are the integral along the ray-path of the slowness. But this tomography like problem is nonlinear, mainly because the unknown ray-path geometry depends upon the velocity (or slowness) distribution. In spite of its nonlinear character, travel-time tomography turned out to be very effective for determining the propagation velocity in the Earth or in the subsurface, the latter aspect being a key element for seismic imaging, in particular using methods mentioned in Section "Diffraction tomography".

=== Mathematical aspects: Hadamard's questions ===

The questions concern well-posedness: Does the least-squares problem have a unique solution which depends continuously on the data (stability problem)? It is the first question, but it is also a difficult one because of the non-linearity of <math>F</math>. 
In order to see where the difficulties arise from, Chavent<ref name=Chavent>{{cite book |last1=Chavent |first1=Guy |title=Nonlinear Least Squares for Inverse problems |date=2010 |publisher=Springer |isbn=978-90-481-2785-6}}</ref> proposed to conceptually split the minimization of the data misfit function into two consecutive steps (<math>P_\text{adm}</math> is the subset of admissible models):
* projection step: given <math>d_\text{obs}</math> find a projection on <math>F(P_\text{adm})</math> (nearest point on <math>F(P_\text{adm})</math> according to the distance involved in the definition of the objective function)
* given this projection find one pre-image that is a model whose image by operator <math>F</math> is this projection.
Difficulties can - and usually will - arise in both steps:
# operator <math>F</math> is not likely to be one-to-one, therefore there can be more than one pre-image,
# even when <math>F</math> is one-to-one, its inverse may not be continuous over <math>F(P)</math>,
# the projection on <math>F(P_\text{adm})</math> may not exist, should this set be not closed,
# the projection on <math>F(P_\text{adm})</math> can be non-unique and not continuous as this can be non-convex due to the non-linearity of <math>F</math>.
We refer to Chavent<ref name=Chavent/> for a mathematical analysis of these points.

=== Computational aspects ===

==== A non-convex data misfit function ====
The forward map being nonlinear, the data misfit function is likely to be non-convex, making local minimization techniques inefficient. Several approaches have been investigated to overcome this difficulty: 
* use of global optimization techniques such as sampling of the posterior density function and [[Metropolis–Hastings algorithm|Metropolis algorithm]] in the inverse problem probabilistic framework,<ref>{{cite journal |last1=Koren |first1=Zvi |last2=Mosegaard |first2=Klaus |last3=Landa |first3=Evgeny |last4=Thore |first4=Pierre |last5=Tarantola |first5=Albert |title=Monte Carlo Estimation and Resolution Analysis of Seismic Background Velocities |journal=Journal of Geophysical Research |date=1991 |volume=96 |issue=B12|pages=20289–20299 |doi=10.1029/91JB02278 |bibcode=1991JGR....9620289K }}</ref> genetic algorithms (alone or in combination with Metropolis algorithm: see<ref>{{cite journal| last1=Tahmasebi|first1=Pejman| last2=Javadpour|first2=Farzam| last3=Sahimi|first3=Muhammad| title=Stochastic shale permeability matching: Three-dimensional characterization and modeling| journal=International Journal of Coal Geology| date=August 2016| volume=165| pages=231–242| doi=10.1016/j.coal.2016.08.024|bibcode=2016IJCG..165..231T | url=https://www.researchgate.net/publication/307626119}}</ref> for an application to the determination of permeabilities that match the existing permeability data), neural networks, regularization techniques including multi scale analysis;
* reformulation of the least-squares objective function so as to make it smoother (see<ref name="ReferenceB"/><ref name="ReferenceC"/> for the inverse problem in the wave equations.)

==== Computation of the gradient of the objective function ====

Inverse problems, especially in infinite dimension, may be large size, thus requiring important computing time. When the forward map is nonlinear, the computational difficulties increase and minimizing the objective function can be difficult. Contrary to the linear situation, an explicit use of the Hessian matrix for solving the normal equations does not make sense here: the Hessian matrix varies with models. Much more effective is the evaluation of the gradient of the objective function for some models. Important computational effort can be saved when we can avoid the very heavy computation of the [[Jacobian matrix and determinant|Jacobian]] (often called "[[Fréchet derivative]]s"): the adjoint state method, proposed by Chavent and Lions,<ref>{{cite book |last1=Chavent |first1=Guy |title=Identification de coefficients répartis dans les équations aux dérivées partielles |date=1971 |publisher=Thèse d'Etat |location=Université Paris 6}}</ref> is aimed to avoid this very heavy computation. It is now very widely used.<ref>{{cite journal |last1=Plessix |first1=René |title=A review of the adjoint-state method for computing the gradient of a functional with geophysical applications |journal=Geophysical Journal International |date=2006 |volume=167 |issue=2 |pages=495–503 |doi=10.1111/j.1365-246X.2006.02978.x |bibcode=2006GeoJI.167..495P |doi-access=free }}</ref>
[[File:Inversion algorithm.jpg|thumb|An inversion algorithm (published Under a Creative Commons license, CC BY-NC-ND by Elsevier)<ref name=":0" />]]

==Applications==
Inverse problem theory is used extensively in weather predictions, oceanography, hydrology, [[Medicine:Inverse recovery in EEG|neuroscience]], and petroleum engineering.<ref>{{Cite journal |last1=Grech |first1=Roberta |last2=Cassar |first2=Tracey |last3=Muscat |first3=Joseph |last4=Camilleri |first4=Kenneth P. |last5=Fabri |first5=Simon G. |last6=Zervakis |first6=Michalis |last7=Xanthopoulos |first7=Petros |last8=Sakkalis |first8=Vangelis |last9=Vanrumste |first9=Bart |date=2008-11-07 |title=Review on solving the inverse problem in EEG source analysis |journal=Journal of Neuroengineering and Rehabilitation |volume=5 |article-number=25 |doi=10.1186/1743-0003-5-25 |doi-access=free |issn=1743-0003 |pmc=2605581 |pmid=18990257}}</ref><ref name="Wunsch1996">{{cite book| author=Carl Wunsch| title=The Ocean Circulation Inverse Problem| url=https://books.google.com/books?id=ugHsLF1RNacC&pg=PR9| date=13 June 1996| publisher=Cambridge University Press| isbn=978-0-521-48090-1|pages=9–}}</ref><ref>{{cite journal| last1=Tahmasebi|first1=Pejman| last2=Javadpour|first2=Farzam| last3=Sahimi|first3=Muhammad| title=Stochastic shale permeability matching: Three-dimensional characterization and modeling| journal=International Journal of Coal Geology| date=August 2016| volume=165| pages=231–242| doi=10.1016/j.coal.2016.08.024|bibcode=2016IJCG..165..231T }}</ref><ref>{{Cite journal| last1=Knighton|first1=James| last2=Singh|first2=Kanishka| last3=Evaristo|first3=Jaivime| date=2020| title=Understanding Catchment-Scale Forest Root Water Uptake Strategies Across the Continental United States Through Inverse Ecohydrological Modeling | journal=Geophysical Research Letters |language=en|volume=47|issue=1 | article-number=e2019GL085937 | doi=10.1029/2019GL085937| bibcode=2020GeoRL..4785937K| s2cid=213914582| issn=1944-8007| doi-access=free}}</ref> Another application is inversion of elastic waves for non-destructive characterization of engineering structures.<ref name=":0">{{Cite journal |last1=Taslimian |first1=Rohollah |last2=Jaganathan |first2=Arun P. |date=2024 |title=Inversion of circumferential elastic waves for characterization of concrete pipes |journal=Wave Motion |language=en |volume=127 |article-number=103272 |doi=10.1016/j.wavemoti.2024.103272|doi-access=free |bibcode=2024WaMot.12703272T }}</ref>

Inverse problems are also found in the field of heat transfer, where a surface heat flux<ref name="Figueiredo2014">{{cite book|author=Patric Figueiredo|title=Development Of An Iterative Method For Solving Multidimensional Inverse Heat Conduction Problems|url=https://www.academia.edu/9823088|date=December 2014|publisher=Lehrstuhl für Wärme- und Stoffübertragung RWTH Aachen}} </ref> is estimated outgoing from temperature data measured inside a rigid body; and, in understanding the controls on plant-matter decay.<ref>{{Cite journal| last1=Forney|first1=David C.| last2=Rothman|first2=Daniel H.| date=2012-09-07 |title=Common structure in the heterogeneity of plant-matter decay |journal=Journal of the Royal Society Interface |volume=9|issue=74|pages=2255–2267 |doi=10.1098/rsif.2012.0122 |pmc=3405759 |pmid=22535699|bibcode=2012JRSI....9.2255F }}</ref> The linear inverse problem is also the fundamental of [[Spectral density estimation|spectral estimation]] and [[Direction of arrival|direction-of-arrival]] (DOA) estimation in [[Signal processing|signal processing]].

[[Inverse lithography]] is used in photomask design for [[Engineering:Semiconductor device fabrication|semiconductor device fabrication]].

== Academic journals ==
Four main academic journals cover inverse problems in general:
*''[[Physics:Inverse Problems|Inverse Problems]]''
*''Journal of Inverse and Ill-posed Problems''<ref>{{cite web |title=Journal of Inverse and Ill-posed Problems |url=http://www.reference-global.com/loi/jiip |archive-url=https://archive.today/20130201045242/http://www.reference-global.com/loi/jiip |archive-date=February 1, 2013}}</ref>
*''Inverse Problems in Science and Engineering''<ref>{{cite web |title=Inverse Problems in Science and Engineering: Vol 25, No 4 |url=http://www.tandf.co.uk/journals/titles/17415977.asp}}</ref>
*''Inverse Problems and Imaging''<ref>{{cite web |title=IPI |url=http://aimsciences.org/journals/ipi/ipi_online.jsp |archive-url=https://web.archive.org/web/20061011090005/http://aimsciences.org/journals/ipi/ipi_online.jsp |archive-date=11 October 2006}}</ref>
Many journals on medical imaging, geophysics, non-destructive testing, etc. are dominated by inverse problems in those areas.

==See also==
*{{annotated link|Backus–Gilbert method}}
*{{annotated link|Compressed sensing}}
**{{annotated link|Physics:Algebraic reconstruction technique}}
**{{annotated link|Iterative reconstruction}}
*{{annotated link|Earth:Data assimilation}}
*{{annotated link|Engineering:Engineering optimization}}
*{{annotated link|Grey box model}}
*{{annotated link|Medicine:Inverse recovery in EEG}}
*{{annotated link|Physics:Mathematical geophysics}}
*{{annotated link|Optimal estimation}}
*{{annotated link|Problem of induction}}
*{{annotated link|Earth:Seismic inversion}}
*{{annotated link|Tikhonov regularization}}

==Notes==
{{Reflist}}

==References==
*Chadan, Khosrow & Sabatier, Pierre Célestin (1977). ''Inverse Problems in Quantum Scattering Theory''. Springer-Verlag. {{ISBN|0-387-08092-9}}
*Aster, Richard; Borchers, Brian, and Thurber, Clifford (2018). ''Parameter Estimation and Inverse Problems'', Third Edition, Elsevier. {{ISBN|9780128134238}}, {{ISBN|9780128134238}}
*{{cite book |last1=Press |first1=WH |last2=Teukolsky |first2=SA |last3=Vetterling |first3=WT |last4=Flannery |first4=BP |year=2007 |title=Numerical Recipes: The Art of Scientific Computing |edition=3rd |publisher=Cambridge University Press |location=New York |isbn=978-0-521-88068-8 |chapter=Section 19.4. Inverse Problems and the Use of A Priori Information |chapter-url=http://apps.nrbook.com/empanel/index.html#pg=1001 |access-date=2011-08-17 |archive-date=2011-08-11 |archive-url=https://web.archive.org/web/20110811154417/http://apps.nrbook.com/empanel/index.html#pg=1001 }}
*Zhdanov, M. S. (2015). ''Inverse Theory and Applications in Geophysics''. Elsevier. {{ISBN|978-0-444-626745-}}

==Further reading==
*{{cite book|author=C. W. Groetsch| title=Inverse Problems: Activities for Undergraduates| year=1999| publisher=Cambridge University Press| isbn=978-0-88385-716-8}}
*{{Cite journal |last=Kirkeby |first=Adrian |date=2024 |title=Feynman's Inverse Problem |url=https://epubs.siam.org/doi/10.1137/23M1611488 |journal=SIAM Review |language=en |volume=66 |issue=4 |pages=694–718 |doi=10.1137/23M1611488 |issn=0036-1445|arxiv=2310.15589 }}

==External links==
*[http://www.inverse-problems.net/ Inverse Problems International Association] {{Webarchive|url=https://web.archive.org/web/20170615131709/http://inverse-problems.net/ |date=2017-06-15 }}
*[http://www.eurasianip.org Eurasian Association on Inverse Problems]
*[http://fips.fi/ Finnish Inverse Problems Society]
*[https://web.archive.org/web/20070925210904/http://www.mth.msu.edu/ipnet/ Inverse Problems Network]
*[http://www.ipgp.jussieu.fr/~tarantola/ Albert Tarantola's website], includes a free PDF version of his Inverse Problem Theory book, and some online articles on Inverse Problems
*[http://www.me.ua.edu/inverse/ Inverse Problems page at the University of Alabama] {{Webarchive|url=https://web.archive.org/web/20140405095149/http://www.me.ua.edu/inverse/ |date=2014-04-05 }}
*[http://imgp.nbi.ku.dk/ Inverse Problems and Geostatistics Project] {{Webarchive|url=https://web.archive.org/web/20171102202207/http://imgp.nbi.ku.dk/ |date=2017-11-02 }}, Niels Bohr Institute, University of Copenhagen
*[http://research.ganse.org/lectures/invtheory Andy Ganse's Geophysical Inverse Theory Resources Page]
*[http://math.tkk.fi/inverse-coe Finnish Centre of Excellence in Inverse Problems Research] {{Webarchive|url=https://web.archive.org/web/20090220183051/http://math.tkk.fi/inverse-coe |date=2009-02-20 }}

{{DEFAULTSORT:Inverse Problem}}
[[Category:Inverse problems| ]]

{{Sourceattribution|Inverse problem}}