Abstract
We consider the problem of minimizing a smooth function of n variables subject to m smooth equality constraints. We begin by describing various approaches to Newton's method for this problem, with emphasis on the recent work of Goodman. This leads to the proposal of a Broyden-type method which updates an n multiplied by (n-m) matrix approximating a 'one-sided projected Hessian' of a Lagrangian function. This method is shown to converge Q-superlinearly. We also give a new short proof of the Boggs-Tolle-Wang necessary and sufficient condition for Q-superlinear convergence of a class of quasi-Newton methods for solving this problem. Finally, we describe an algorithm which updates an approximation to a 'two-sided projected Hessian', a symmetric matrix of order n-m which is generally positive definite near a solution. We present several new variants of this algorithm and show that under certain conditions they all have a local two-step Q-superlinear convergence property, even though only one set of gradients is evaluated per iteration. Numerical results are presented, indicating that the methods may be very useful in practice.
| Original language | English (US) |
|---|---|
| Pages (from-to) | 821-850 |
| Number of pages | 30 |
| Journal | SIAM Journal on Numerical Analysis |
| Volume | 22 |
| Issue number | 2 |
| State | Published - Oct 1985 |
Fingerprint
ASJC Scopus subject areas
- Mathematics(all)
- Applied Mathematics
- Computational Mathematics
Cite this
PROJECTED HESSIAN UPDATING ALGORITHMS FOR NONLINEARLY CONSTRAINED OPTIMIZATION. / Nocedal, Jorge; Overton, Michael L.
In: SIAM Journal on Numerical Analysis, Vol. 22, No. 2, 10.1985, p. 821-850.Research output: Contribution to journal › Article
}
TY - JOUR
T1 - PROJECTED HESSIAN UPDATING ALGORITHMS FOR NONLINEARLY CONSTRAINED OPTIMIZATION.
AU - Nocedal, Jorge
AU - Overton, Michael L.
PY - 1985/10
Y1 - 1985/10
N2 - We consider the problem of minimizing a smooth function of n variables subject to m smooth equality constraints. We begin by describing various approaches to Newton's method for this problem, with emphasis on the recent work of Goodman. This leads to the proposal of a Broyden-type method which updates an n multiplied by (n-m) matrix approximating a 'one-sided projected Hessian' of a Lagrangian function. This method is shown to converge Q-superlinearly. We also give a new short proof of the Boggs-Tolle-Wang necessary and sufficient condition for Q-superlinear convergence of a class of quasi-Newton methods for solving this problem. Finally, we describe an algorithm which updates an approximation to a 'two-sided projected Hessian', a symmetric matrix of order n-m which is generally positive definite near a solution. We present several new variants of this algorithm and show that under certain conditions they all have a local two-step Q-superlinear convergence property, even though only one set of gradients is evaluated per iteration. Numerical results are presented, indicating that the methods may be very useful in practice.
AB - We consider the problem of minimizing a smooth function of n variables subject to m smooth equality constraints. We begin by describing various approaches to Newton's method for this problem, with emphasis on the recent work of Goodman. This leads to the proposal of a Broyden-type method which updates an n multiplied by (n-m) matrix approximating a 'one-sided projected Hessian' of a Lagrangian function. This method is shown to converge Q-superlinearly. We also give a new short proof of the Boggs-Tolle-Wang necessary and sufficient condition for Q-superlinear convergence of a class of quasi-Newton methods for solving this problem. Finally, we describe an algorithm which updates an approximation to a 'two-sided projected Hessian', a symmetric matrix of order n-m which is generally positive definite near a solution. We present several new variants of this algorithm and show that under certain conditions they all have a local two-step Q-superlinear convergence property, even though only one set of gradients is evaluated per iteration. Numerical results are presented, indicating that the methods may be very useful in practice.
UR - http://www.scopus.com/inward/record.url?scp=0022145863&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=0022145863&partnerID=8YFLogxK
M3 - Article
VL - 22
SP - 821
EP - 850
JO - SIAM Journal on Numerical Analysis
JF - SIAM Journal on Numerical Analysis
SN - 0036-1429
IS - 2
ER -