PROJECTED HESSIAN UPDATING ALGORITHMS FOR NONLINEARLY CONSTRAINED OPTIMIZATION.

Jorge Nocedal, Michael L. Overton

Research output: Contribution to journalArticle

Abstract

We consider the problem of minimizing a smooth function of n variables subject to m smooth equality constraints. We begin by describing various approaches to Newton's method for this problem, with emphasis on the recent work of Goodman. This leads to the proposal of a Broyden-type method which updates an n multiplied by (n-m) matrix approximating a 'one-sided projected Hessian' of a Lagrangian function. This method is shown to converge Q-superlinearly. We also give a new short proof of the Boggs-Tolle-Wang necessary and sufficient condition for Q-superlinear convergence of a class of quasi-Newton methods for solving this problem. Finally, we describe an algorithm which updates an approximation to a 'two-sided projected Hessian', a symmetric matrix of order n-m which is generally positive definite near a solution. We present several new variants of this algorithm and show that under certain conditions they all have a local two-step Q-superlinear convergence property, even though only one set of gradients is evaluated per iteration. Numerical results are presented, indicating that the methods may be very useful in practice.

Original languageEnglish (US)
Pages (from-to)821-850
Number of pages30
JournalSIAM Journal on Numerical Analysis
Volume22
Issue number2
StatePublished - Oct 1985

Fingerprint

Constrained optimization
Newton-Raphson method
Constrained Optimization
Updating
Superlinear Convergence
Update
Quasi-Newton Method
Equality Constraints
Symmetric matrix
Smooth function
Newton Methods
Positive definite
Convergence Properties
Gradient
Converge
Iteration
Necessary Conditions
Numerical Results
Sufficient Conditions
Approximation

ASJC Scopus subject areas

  • Mathematics(all)
  • Applied Mathematics
  • Computational Mathematics

Cite this

PROJECTED HESSIAN UPDATING ALGORITHMS FOR NONLINEARLY CONSTRAINED OPTIMIZATION. / Nocedal, Jorge; Overton, Michael L.

In: SIAM Journal on Numerical Analysis, Vol. 22, No. 2, 10.1985, p. 821-850.

Research output: Contribution to journalArticle

@article{64ff539262944cc8b3e48adb0599d6f5,
title = "PROJECTED HESSIAN UPDATING ALGORITHMS FOR NONLINEARLY CONSTRAINED OPTIMIZATION.",
abstract = "We consider the problem of minimizing a smooth function of n variables subject to m smooth equality constraints. We begin by describing various approaches to Newton's method for this problem, with emphasis on the recent work of Goodman. This leads to the proposal of a Broyden-type method which updates an n multiplied by (n-m) matrix approximating a 'one-sided projected Hessian' of a Lagrangian function. This method is shown to converge Q-superlinearly. We also give a new short proof of the Boggs-Tolle-Wang necessary and sufficient condition for Q-superlinear convergence of a class of quasi-Newton methods for solving this problem. Finally, we describe an algorithm which updates an approximation to a 'two-sided projected Hessian', a symmetric matrix of order n-m which is generally positive definite near a solution. We present several new variants of this algorithm and show that under certain conditions they all have a local two-step Q-superlinear convergence property, even though only one set of gradients is evaluated per iteration. Numerical results are presented, indicating that the methods may be very useful in practice.",
author = "Jorge Nocedal and Overton, {Michael L.}",
year = "1985",
month = "10",
language = "English (US)",
volume = "22",
pages = "821--850",
journal = "SIAM Journal on Numerical Analysis",
issn = "0036-1429",
publisher = "Society for Industrial and Applied Mathematics Publications",
number = "2",

}

TY - JOUR

T1 - PROJECTED HESSIAN UPDATING ALGORITHMS FOR NONLINEARLY CONSTRAINED OPTIMIZATION.

AU - Nocedal, Jorge

AU - Overton, Michael L.

PY - 1985/10

Y1 - 1985/10

N2 - We consider the problem of minimizing a smooth function of n variables subject to m smooth equality constraints. We begin by describing various approaches to Newton's method for this problem, with emphasis on the recent work of Goodman. This leads to the proposal of a Broyden-type method which updates an n multiplied by (n-m) matrix approximating a 'one-sided projected Hessian' of a Lagrangian function. This method is shown to converge Q-superlinearly. We also give a new short proof of the Boggs-Tolle-Wang necessary and sufficient condition for Q-superlinear convergence of a class of quasi-Newton methods for solving this problem. Finally, we describe an algorithm which updates an approximation to a 'two-sided projected Hessian', a symmetric matrix of order n-m which is generally positive definite near a solution. We present several new variants of this algorithm and show that under certain conditions they all have a local two-step Q-superlinear convergence property, even though only one set of gradients is evaluated per iteration. Numerical results are presented, indicating that the methods may be very useful in practice.

AB - We consider the problem of minimizing a smooth function of n variables subject to m smooth equality constraints. We begin by describing various approaches to Newton's method for this problem, with emphasis on the recent work of Goodman. This leads to the proposal of a Broyden-type method which updates an n multiplied by (n-m) matrix approximating a 'one-sided projected Hessian' of a Lagrangian function. This method is shown to converge Q-superlinearly. We also give a new short proof of the Boggs-Tolle-Wang necessary and sufficient condition for Q-superlinear convergence of a class of quasi-Newton methods for solving this problem. Finally, we describe an algorithm which updates an approximation to a 'two-sided projected Hessian', a symmetric matrix of order n-m which is generally positive definite near a solution. We present several new variants of this algorithm and show that under certain conditions they all have a local two-step Q-superlinear convergence property, even though only one set of gradients is evaluated per iteration. Numerical results are presented, indicating that the methods may be very useful in practice.

UR - http://www.scopus.com/inward/record.url?scp=0022145863&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0022145863&partnerID=8YFLogxK

M3 - Article

AN - SCOPUS:0022145863

VL - 22

SP - 821

EP - 850

JO - SIAM Journal on Numerical Analysis

JF - SIAM Journal on Numerical Analysis

SN - 0036-1429

IS - 2

ER -