Boosting with abstention

Corinna Cortes, Giulia De Salvo, Mehryar Mohri

Research output: Contribution to journalArticle

Abstract

We present a new boosting algorithm for the key scenario of binary classification with abstention where the algorithm can abstain from predicting the label of a point, at the price of a fixed cost. At each round, our algorithm selects a pair of functions, a base predictor and a base abstention function. We define convex upper bounds for the natural loss function associated to this problem, which we prove to be calibrated with respect to the Bayes solution. Our algorithm benefits from general margin-based learning guarantees which we derive for ensembles of pairs of base predictor and abstention functions, in terms of the Rademacher complexities of the corresponding function classes. We give convergence guarantees for our algorithm along with a linear-time weak-learning algorithm for abstention stumps. We also report the results of several experiments suggesting that our algorithm provides a significant improvement in practice over two confidence-based algorithms.

Original languageEnglish (US)
Pages (from-to)1668-1676
Number of pages9
JournalAdvances in Neural Information Processing Systems
StatePublished - 2016

Fingerprint

Learning algorithms
Labels
Costs
Experiments

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Information Systems
  • Signal Processing

Cite this

Boosting with abstention. / Cortes, Corinna; De Salvo, Giulia; Mohri, Mehryar.

In: Advances in Neural Information Processing Systems, 2016, p. 1668-1676.

Research output: Contribution to journalArticle

Cortes, Corinna ; De Salvo, Giulia ; Mohri, Mehryar. / Boosting with abstention. In: Advances in Neural Information Processing Systems. 2016 ; pp. 1668-1676.
@article{d868f105f4b54999b38524ee32da91bf,
title = "Boosting with abstention",
abstract = "We present a new boosting algorithm for the key scenario of binary classification with abstention where the algorithm can abstain from predicting the label of a point, at the price of a fixed cost. At each round, our algorithm selects a pair of functions, a base predictor and a base abstention function. We define convex upper bounds for the natural loss function associated to this problem, which we prove to be calibrated with respect to the Bayes solution. Our algorithm benefits from general margin-based learning guarantees which we derive for ensembles of pairs of base predictor and abstention functions, in terms of the Rademacher complexities of the corresponding function classes. We give convergence guarantees for our algorithm along with a linear-time weak-learning algorithm for abstention stumps. We also report the results of several experiments suggesting that our algorithm provides a significant improvement in practice over two confidence-based algorithms.",
author = "Corinna Cortes and {De Salvo}, Giulia and Mehryar Mohri",
year = "2016",
language = "English (US)",
pages = "1668--1676",
journal = "Advances in Neural Information Processing Systems",
issn = "1049-5258",

}

TY - JOUR

T1 - Boosting with abstention

AU - Cortes, Corinna

AU - De Salvo, Giulia

AU - Mohri, Mehryar

PY - 2016

Y1 - 2016

N2 - We present a new boosting algorithm for the key scenario of binary classification with abstention where the algorithm can abstain from predicting the label of a point, at the price of a fixed cost. At each round, our algorithm selects a pair of functions, a base predictor and a base abstention function. We define convex upper bounds for the natural loss function associated to this problem, which we prove to be calibrated with respect to the Bayes solution. Our algorithm benefits from general margin-based learning guarantees which we derive for ensembles of pairs of base predictor and abstention functions, in terms of the Rademacher complexities of the corresponding function classes. We give convergence guarantees for our algorithm along with a linear-time weak-learning algorithm for abstention stumps. We also report the results of several experiments suggesting that our algorithm provides a significant improvement in practice over two confidence-based algorithms.

AB - We present a new boosting algorithm for the key scenario of binary classification with abstention where the algorithm can abstain from predicting the label of a point, at the price of a fixed cost. At each round, our algorithm selects a pair of functions, a base predictor and a base abstention function. We define convex upper bounds for the natural loss function associated to this problem, which we prove to be calibrated with respect to the Bayes solution. Our algorithm benefits from general margin-based learning guarantees which we derive for ensembles of pairs of base predictor and abstention functions, in terms of the Rademacher complexities of the corresponding function classes. We give convergence guarantees for our algorithm along with a linear-time weak-learning algorithm for abstention stumps. We also report the results of several experiments suggesting that our algorithm provides a significant improvement in practice over two confidence-based algorithms.

UR - http://www.scopus.com/inward/record.url?scp=85018915857&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85018915857&partnerID=8YFLogxK

M3 - Article

AN - SCOPUS:85018915857

SP - 1668

EP - 1676

JO - Advances in Neural Information Processing Systems

JF - Advances in Neural Information Processing Systems

SN - 1049-5258

ER -