Duality gap

In optimization problems in applied mathematics, the duality gap is the difference between the primal and dual solutions. If [math]\displaystyle{ d^* }[/math] is the optimal dual value and [math]\displaystyle{ p^* }[/math] is the optimal primal value then the duality gap is equal to [math]\displaystyle{ p^* - d^* }[/math]. This value is always greater than or equal to 0 (for minimization problems). The duality gap is zero if and only if strong duality holds. Otherwise the gap is strictly positive and weak duality holds. In general given two dual pairs separated locally convex spaces [math]\displaystyle{ \left(X,X^*\right) }[/math] and [math]\displaystyle{ \left(Y,Y^*\right) }[/math]. Then given the function [math]\displaystyle{ f: X \to \mathbb{R} \cup \{+\infty\} }[/math], we can define the primal problem by

[math]\displaystyle{ \inf_{x \in X} f(x). \, }[/math]

If there are constraint conditions, these can be built into the function [math]\displaystyle{ f }[/math] by letting [math]\displaystyle{ f = f + I_\text{constraints} }[/math] where [math]\displaystyle{ I }[/math] is the indicator function. Then let [math]\displaystyle{ F: X \times Y \to \mathbb{R} \cup \{+\infty\} }[/math] be a perturbation function such that [math]\displaystyle{ F(x,0) = f(x) }[/math]. The duality gap is the difference given by

[math]\displaystyle{ \inf_{x \in X} [F(x,0)] - \sup_{y^* \in Y^*} [-F^*(0,y^*)] }[/math]

where [math]\displaystyle{ F^* }[/math] is the convex conjugate in both variables.

In computational optimization, another "duality gap" is often reported, which is the difference in value between any dual solution and the value of a feasible but suboptimal iterate for the primal problem. This alternative "duality gap" quantifies the discrepancy between the value of a current feasible but suboptimal iterate for the primal problem and the value of the dual problem; the value of the dual problem is, under regularity conditions, equal to the value of the convex relaxation of the primal problem: The convex relaxation is the problem arising replacing a non-convex feasible set with its closed convex hull and with replacing a non-convex function with its convex closure, that is the function that has the epigraph that is the closed convex hull of the original primal objective function.