Analysis of the gradient method with an Armijo–Wolfe line search on a class of non-smooth convex functions

Research output: Contribution to journalArticle

Abstract

It has long been known that the gradient (steepest descent) method may fail on non-smooth problems, but the examples that have appeared in the literature are either devised specifically to defeat a gradient or subgradient method with an exact line search or are unstable with respect to perturbation of the initial point. We give an analysis of the gradient method with steplengths satisfying the Armijo and Wolfe inexact line search conditions on the non-smooth convex function (Formula presented.). We show that if a is sufficiently large, satisfying a condition that depends only on the Armijo parameter, then, when the method is initiated at any point (Formula presented.) with (Formula presented.), the iterates converge to a point (Formula presented.) with (Formula presented.), although f is unbounded below. We also give conditions under which the iterates (Formula presented.), using a specific Armijo–Wolfe bracketing line search. Our experimental results demonstrate that our analysis is reasonably tight.

Original languageEnglish (US)
Pages (from-to)223-242
Number of pages20
JournalOptimization Methods and Software
Volume35
Issue number2
DOIs
StatePublished - Mar 3 2020

Keywords

  • Steepest descent method
  • convex optimization
  • non-smooth optimization

ASJC Scopus subject areas

  • Software
  • Control and Optimization
  • Applied Mathematics

Fingerprint Dive into the research topics of 'Analysis of the gradient method with an Armijo–Wolfe line search on a class of non-smooth convex functions'. Together they form a unique fingerprint.

  • Cite this