NEWUOA

NEWUOA is a numerical optimization algorithm by Michael J. D. Powell. It is also the name of Powell's Fortran 77 implementation of the algorithm. NEWUOA and all the other derivative-free optimization solvers of Powell's are included in PDFO, which provides MATLAB and Python interfaces for using these solvers on Linux, Mac, and Windows. NEWUOA solves unconstrained optimization problems without using derivatives, which makes it a derivative-free algorithm. The algorithm is iterative and exploits trust-region technique. On each iteration, the algorithm establishes a model function [math]\displaystyle{ Q_k }[/math] by quadratic interpolation and then minimizes [math]\displaystyle{ Q_k }[/math] within a trust region.

One important feature of NEWUOA algorithm is the least Frobenius norm updating technique. Suppose that the objective function [math]\displaystyle{ f }[/math] has [math]\displaystyle{ n }[/math] variables, and one wants to uniquely determine the quadratic model [math]\displaystyle{ Q_k }[/math] by purely interpolating the function values of [math]\displaystyle{ f }[/math], then it is necessary to evaluate [math]\displaystyle{ f }[/math] at [math]\displaystyle{ (n+1)(n+2)/2 }[/math] points, as a quadratic polynomial of [math]\displaystyle{ n }[/math] variables has this amount of independent coefficients. But this is impractical when [math]\displaystyle{ n }[/math] is large, because the function values are supposed to be expensive in derivative-free optimization. In NEWUOA, the model [math]\displaystyle{ Q_k }[/math] interpolates only [math]\displaystyle{ m }[/math] (an integer between [math]\displaystyle{ n+2 }[/math] and [math]\displaystyle{ (n+1)(n+2)/2 }[/math], typically of order [math]\displaystyle{ n }[/math]) function values of [math]\displaystyle{ f }[/math], and the remaining degrees of freedom are taken up by minimizing the Frobenius norm of [math]\displaystyle{ \nabla^2 Q_k -\nabla^2 Q_{k-1} }[/math]. This technique mimics the least-change secant updates for quasi-Newton methods and can be considered as the derivative-free version of PSB update (Powell's symmetric Broyden update).

To construct the models, NEWUOA maintains a set of interpolation points throughout the iterations. The update of this set is another feature of NEWUOA.

NEWUOA algorithm was developed from UOBYQA (Unconstrained Optimization BY Quadratic Approximation). A major difference between them is that UOBYQA constructs quadratic models by interpolating the objective function at [math]\displaystyle{ (n+1)(n+2)/2 }[/math] points.

NEWUOA software was released on December 16, 2004. It can solve unconstrained optimization problems of a few hundreds variables to high precision without using derivatives. In the software, [math]\displaystyle{ m }[/math] is set to [math]\displaystyle{ 2n+1 }[/math] by default.

Other derivative-free optimization algorithms by Powell include COBYLA, UOBYQA, BOBYQA, and LINCOA. BOBYQA and LINCOA are extensions of NEWUOA to bound constrained and linearly constrained optimization respectively.

Powell did not explain how he coined the name "NEWUOA" either in the introducing report nor in the software, although COBYLA, UOBYQA, BOBYQA and LINCOA are all named by acronyms. Probably "NEWUOA" means "NEW Unconstrained Optimization Algorithm".

NEWUOA software is distributed under The GNU Lesser General Public License (LGPL).

See also

• TOLMIN
• COBYLA
• UOBYQA
• BOBYQA
• LINCOA

External links

• PDFO — Powell's derivative-free optimization solvers (MATLAB and Python interfaces for Powell's Fortran solvers)
• Optimization software by Professor M. J. D. Powell at CCPForge
• M.Powells f77-coded newuoa ported to Fortran 90/95 with easy user interface