Ill-conditioning and computational error in interior methods for nonlinear programming

Research output: Contribution to journalArticle

Abstract

Ill-conditioning has long been regarded as a plague on interior methods, but its damaging effects have rarely been documented. In fact, implementors of interior methods who ignore warnings about the dire consequences of ill-conditioning usually manage to compute accurate solutions. We offer some insight into this seeming contradiction by analyzing ill-conditioning within a primal-dual method in which the full, usually well-conditioned primal-dual matrix is transformed to a "condensed," inherently ill-conditioned matrix Mpd. We show that ill-conditioning in the exact condensed matrix closely resembles that known for the primal barrier Hessian, and then examine the influence of cancellation in the computed constraints. Using the structure of Mpd, various bounds are obtained on the absolute accuracy of the computed primal-dual steps. Without cancellation, the portion of the computed x step in the small space of Mpd (a subspace close to the null space of the Jacobian of the active constraints) has an absolute error bound comparable to machine precision, and its large-space component has a much smaller error bound. With cancellation (the usual case), the absolute error bounds for both the small-and large-space components of the computed x step are comparable to machine precision. In either case, the absolute error bound for the computed multiplier steps associated with active constraints is comparable to machine precision; the computed multiplier steps for inactive constraints, although converging to zero, retain (approximately) full relative precision. Because of errors in forming the right-hand side, the absolute error in the computed solution of the full, well-conditioned primal-dual system is shown to be comparable to machine precision. Thus, under quite general conditions, ill-conditioning in Mpd does not noticeably impair the accuracy of the computed primal-dual steps. (A similar analysis applies to search directions obtained by direct solution of the primal Newton equations.).

Original languageEnglish (US)
Pages (from-to)84-111
Number of pages28
JournalSIAM Journal on Optimization
Volume9
Issue number1
StatePublished - 1998

Fingerprint

Interior Methods
Ill-conditioning
Nonlinear programming
Nonlinear Programming
Primal-dual
Error Bounds
Cancellation
Multiplier
Relative Precision
Primal-dual Method
Null Space
Subspace
Zero

Keywords

  • Barrier method
  • Constrained optimization
  • Interior method
  • Primal-dual method

ASJC Scopus subject areas

  • Mathematics(all)
  • Applied Mathematics

Cite this

Ill-conditioning and computational error in interior methods for nonlinear programming. / Wright, Margaret.

In: SIAM Journal on Optimization, Vol. 9, No. 1, 1998, p. 84-111.

Research output: Contribution to journalArticle

@article{7ef68c69c24545098f0b0e793c66ca00,
title = "Ill-conditioning and computational error in interior methods for nonlinear programming",
abstract = "Ill-conditioning has long been regarded as a plague on interior methods, but its damaging effects have rarely been documented. In fact, implementors of interior methods who ignore warnings about the dire consequences of ill-conditioning usually manage to compute accurate solutions. We offer some insight into this seeming contradiction by analyzing ill-conditioning within a primal-dual method in which the full, usually well-conditioned primal-dual matrix is transformed to a {"}condensed,{"} inherently ill-conditioned matrix Mpd. We show that ill-conditioning in the exact condensed matrix closely resembles that known for the primal barrier Hessian, and then examine the influence of cancellation in the computed constraints. Using the structure of Mpd, various bounds are obtained on the absolute accuracy of the computed primal-dual steps. Without cancellation, the portion of the computed x step in the small space of Mpd (a subspace close to the null space of the Jacobian of the active constraints) has an absolute error bound comparable to machine precision, and its large-space component has a much smaller error bound. With cancellation (the usual case), the absolute error bounds for both the small-and large-space components of the computed x step are comparable to machine precision. In either case, the absolute error bound for the computed multiplier steps associated with active constraints is comparable to machine precision; the computed multiplier steps for inactive constraints, although converging to zero, retain (approximately) full relative precision. Because of errors in forming the right-hand side, the absolute error in the computed solution of the full, well-conditioned primal-dual system is shown to be comparable to machine precision. Thus, under quite general conditions, ill-conditioning in Mpd does not noticeably impair the accuracy of the computed primal-dual steps. (A similar analysis applies to search directions obtained by direct solution of the primal Newton equations.).",
keywords = "Barrier method, Constrained optimization, Interior method, Primal-dual method",
author = "Margaret Wright",
year = "1998",
language = "English (US)",
volume = "9",
pages = "84--111",
journal = "SIAM Journal on Optimization",
issn = "1052-6234",
publisher = "Society for Industrial and Applied Mathematics Publications",
number = "1",

}

TY - JOUR

T1 - Ill-conditioning and computational error in interior methods for nonlinear programming

AU - Wright, Margaret

PY - 1998

Y1 - 1998

N2 - Ill-conditioning has long been regarded as a plague on interior methods, but its damaging effects have rarely been documented. In fact, implementors of interior methods who ignore warnings about the dire consequences of ill-conditioning usually manage to compute accurate solutions. We offer some insight into this seeming contradiction by analyzing ill-conditioning within a primal-dual method in which the full, usually well-conditioned primal-dual matrix is transformed to a "condensed," inherently ill-conditioned matrix Mpd. We show that ill-conditioning in the exact condensed matrix closely resembles that known for the primal barrier Hessian, and then examine the influence of cancellation in the computed constraints. Using the structure of Mpd, various bounds are obtained on the absolute accuracy of the computed primal-dual steps. Without cancellation, the portion of the computed x step in the small space of Mpd (a subspace close to the null space of the Jacobian of the active constraints) has an absolute error bound comparable to machine precision, and its large-space component has a much smaller error bound. With cancellation (the usual case), the absolute error bounds for both the small-and large-space components of the computed x step are comparable to machine precision. In either case, the absolute error bound for the computed multiplier steps associated with active constraints is comparable to machine precision; the computed multiplier steps for inactive constraints, although converging to zero, retain (approximately) full relative precision. Because of errors in forming the right-hand side, the absolute error in the computed solution of the full, well-conditioned primal-dual system is shown to be comparable to machine precision. Thus, under quite general conditions, ill-conditioning in Mpd does not noticeably impair the accuracy of the computed primal-dual steps. (A similar analysis applies to search directions obtained by direct solution of the primal Newton equations.).

AB - Ill-conditioning has long been regarded as a plague on interior methods, but its damaging effects have rarely been documented. In fact, implementors of interior methods who ignore warnings about the dire consequences of ill-conditioning usually manage to compute accurate solutions. We offer some insight into this seeming contradiction by analyzing ill-conditioning within a primal-dual method in which the full, usually well-conditioned primal-dual matrix is transformed to a "condensed," inherently ill-conditioned matrix Mpd. We show that ill-conditioning in the exact condensed matrix closely resembles that known for the primal barrier Hessian, and then examine the influence of cancellation in the computed constraints. Using the structure of Mpd, various bounds are obtained on the absolute accuracy of the computed primal-dual steps. Without cancellation, the portion of the computed x step in the small space of Mpd (a subspace close to the null space of the Jacobian of the active constraints) has an absolute error bound comparable to machine precision, and its large-space component has a much smaller error bound. With cancellation (the usual case), the absolute error bounds for both the small-and large-space components of the computed x step are comparable to machine precision. In either case, the absolute error bound for the computed multiplier steps associated with active constraints is comparable to machine precision; the computed multiplier steps for inactive constraints, although converging to zero, retain (approximately) full relative precision. Because of errors in forming the right-hand side, the absolute error in the computed solution of the full, well-conditioned primal-dual system is shown to be comparable to machine precision. Thus, under quite general conditions, ill-conditioning in Mpd does not noticeably impair the accuracy of the computed primal-dual steps. (A similar analysis applies to search directions obtained by direct solution of the primal Newton equations.).

KW - Barrier method

KW - Constrained optimization

KW - Interior method

KW - Primal-dual method

UR - http://www.scopus.com/inward/record.url?scp=0032237110&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0032237110&partnerID=8YFLogxK

M3 - Article

AN - SCOPUS:0032237110

VL - 9

SP - 84

EP - 111

JO - SIAM Journal on Optimization

JF - SIAM Journal on Optimization

SN - 1052-6234

IS - 1

ER -