Steplength algorithms for minimizing a class of nondifferentiable functions

W. Murray, M. L. Overton

Research output: Contribution to journalArticle

Abstract

Four steplength algorithms are presented for minimizing a class of nondifferentiable functions which includes functions arising from l1 and l approximation problems and penalty functions arising from constrained optimization problems. Two algorithms are given for the case when derivatives are available wherever they exist and two for the case when they are not avaible. We take the view that although a simple steplength algorithm may be all that is required to meet convergence criteria for the overall algorithm, from the point, of view of efficiency it is important that the step achieve as large a reduction in the function value as possible, given a certain limit on the effort to be expended. The algorithms include the facility for varying this limit, producing, anything from an algorithm requiring a single function evaluation to one doing an exact linear search. They are based on univariate minimization algorithms which we present first. These are normally at least quadratically convergent when derivatives are used and superlinearly convergent otherwise, regardless of whether or not the function is differentiable at the minimum.

Original languageEnglish (US)
Pages (from-to)309-331
Number of pages23
JournalComputing
Volume23
Issue number4
DOIs
StatePublished - Dec 1979

Fingerprint

Linear search
Derivatives
Derivative
Convergence Criteria
Function evaluation
Constrained optimization
Approximation Problem
Penalty Function
Evaluation Function
Constrained Optimization Problem
Class
Value Function
Univariate
Differentiable

ASJC Scopus subject areas

  • Computational Theory and Mathematics
  • Theoretical Computer Science

Cite this

Steplength algorithms for minimizing a class of nondifferentiable functions. / Murray, W.; Overton, M. L.

In: Computing, Vol. 23, No. 4, 12.1979, p. 309-331.

Research output: Contribution to journalArticle

@article{b357a32a2e4943b3a689498aca608708,
title = "Steplength algorithms for minimizing a class of nondifferentiable functions",
abstract = "Four steplength algorithms are presented for minimizing a class of nondifferentiable functions which includes functions arising from l1 and l∞ approximation problems and penalty functions arising from constrained optimization problems. Two algorithms are given for the case when derivatives are available wherever they exist and two for the case when they are not avaible. We take the view that although a simple steplength algorithm may be all that is required to meet convergence criteria for the overall algorithm, from the point, of view of efficiency it is important that the step achieve as large a reduction in the function value as possible, given a certain limit on the effort to be expended. The algorithms include the facility for varying this limit, producing, anything from an algorithm requiring a single function evaluation to one doing an exact linear search. They are based on univariate minimization algorithms which we present first. These are normally at least quadratically convergent when derivatives are used and superlinearly convergent otherwise, regardless of whether or not the function is differentiable at the minimum.",
author = "W. Murray and Overton, {M. L.}",
year = "1979",
month = "12",
doi = "10.1007/BF02254861",
language = "English (US)",
volume = "23",
pages = "309--331",
journal = "Computing (Vienna/New York)",
issn = "0010-485X",
publisher = "Springer Wien",
number = "4",

}

TY - JOUR

T1 - Steplength algorithms for minimizing a class of nondifferentiable functions

AU - Murray, W.

AU - Overton, M. L.

PY - 1979/12

Y1 - 1979/12

N2 - Four steplength algorithms are presented for minimizing a class of nondifferentiable functions which includes functions arising from l1 and l∞ approximation problems and penalty functions arising from constrained optimization problems. Two algorithms are given for the case when derivatives are available wherever they exist and two for the case when they are not avaible. We take the view that although a simple steplength algorithm may be all that is required to meet convergence criteria for the overall algorithm, from the point, of view of efficiency it is important that the step achieve as large a reduction in the function value as possible, given a certain limit on the effort to be expended. The algorithms include the facility for varying this limit, producing, anything from an algorithm requiring a single function evaluation to one doing an exact linear search. They are based on univariate minimization algorithms which we present first. These are normally at least quadratically convergent when derivatives are used and superlinearly convergent otherwise, regardless of whether or not the function is differentiable at the minimum.

AB - Four steplength algorithms are presented for minimizing a class of nondifferentiable functions which includes functions arising from l1 and l∞ approximation problems and penalty functions arising from constrained optimization problems. Two algorithms are given for the case when derivatives are available wherever they exist and two for the case when they are not avaible. We take the view that although a simple steplength algorithm may be all that is required to meet convergence criteria for the overall algorithm, from the point, of view of efficiency it is important that the step achieve as large a reduction in the function value as possible, given a certain limit on the effort to be expended. The algorithms include the facility for varying this limit, producing, anything from an algorithm requiring a single function evaluation to one doing an exact linear search. They are based on univariate minimization algorithms which we present first. These are normally at least quadratically convergent when derivatives are used and superlinearly convergent otherwise, regardless of whether or not the function is differentiable at the minimum.

UR - http://www.scopus.com/inward/record.url?scp=0018704971&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0018704971&partnerID=8YFLogxK

U2 - 10.1007/BF02254861

DO - 10.1007/BF02254861

M3 - Article

VL - 23

SP - 309

EP - 331

JO - Computing (Vienna/New York)

JF - Computing (Vienna/New York)

SN - 0010-485X

IS - 4

ER -