Learning with rejection

Corinna Cortes, Giulia DeSalvo, Mehryar Mohri

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We introduce a novel framework for classification with a rejection option that consists of simultaneously learning two functions: a classifier along with a rejection function. We present a full theoretical analysis of this framework including new data-dependent learning bounds in terms of the Rademacher complexities of the classifier and rejection families as well as consistency and calibration results. These theoretical guarantees guide us in designing new algorithms that can exploit different kernel-based hypothesis sets for the classifier and rejection functions. We compare and contrast our general framework with the special case of confidence-based rejection for which we devise alternative loss functions and algorithms as well. We report the results of several experiments showing that our kernel-based algorithms can yield a notable improvement over the best existing confidence-based rejection algorithm.

Original languageEnglish (US)
Title of host publicationAlgorithmic Learning Theory - 27th International Conference, ALT 2016, Proceedings
PublisherSpringer Verlag
Pages67-82
Number of pages16
Volume9925 LNAI
ISBN (Print)9783319463780
DOIs
StatePublished - 2016
Event27th International Conference on Algorithmic Learning Theory, ALT 2016 - Bari, Italy
Duration: Oct 19 2016Oct 21 2016

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume9925 LNAI
ISSN (Print)03029743
ISSN (Electronic)16113349

Other

Other27th International Conference on Algorithmic Learning Theory, ALT 2016
CountryItaly
CityBari
Period10/19/1610/21/16

Fingerprint

Rejection
Classifiers
Classifier
Confidence
kernel
Dependent Data
Calibration
Loss Function
Learning
Theoretical Analysis
Experiments
Alternatives
Experiment
Framework

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Cite this

Cortes, C., DeSalvo, G., & Mohri, M. (2016). Learning with rejection. In Algorithmic Learning Theory - 27th International Conference, ALT 2016, Proceedings (Vol. 9925 LNAI, pp. 67-82). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9925 LNAI). Springer Verlag. https://doi.org/10.1007/978-3-319-46379-7_5

Learning with rejection. / Cortes, Corinna; DeSalvo, Giulia; Mohri, Mehryar.

Algorithmic Learning Theory - 27th International Conference, ALT 2016, Proceedings. Vol. 9925 LNAI Springer Verlag, 2016. p. 67-82 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9925 LNAI).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Cortes, C, DeSalvo, G & Mohri, M 2016, Learning with rejection. in Algorithmic Learning Theory - 27th International Conference, ALT 2016, Proceedings. vol. 9925 LNAI, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 9925 LNAI, Springer Verlag, pp. 67-82, 27th International Conference on Algorithmic Learning Theory, ALT 2016, Bari, Italy, 10/19/16. https://doi.org/10.1007/978-3-319-46379-7_5
Cortes C, DeSalvo G, Mohri M. Learning with rejection. In Algorithmic Learning Theory - 27th International Conference, ALT 2016, Proceedings. Vol. 9925 LNAI. Springer Verlag. 2016. p. 67-82. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-319-46379-7_5
Cortes, Corinna ; DeSalvo, Giulia ; Mohri, Mehryar. / Learning with rejection. Algorithmic Learning Theory - 27th International Conference, ALT 2016, Proceedings. Vol. 9925 LNAI Springer Verlag, 2016. pp. 67-82 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{9115b6c49f784e5191a2228055640de0,
title = "Learning with rejection",
abstract = "We introduce a novel framework for classification with a rejection option that consists of simultaneously learning two functions: a classifier along with a rejection function. We present a full theoretical analysis of this framework including new data-dependent learning bounds in terms of the Rademacher complexities of the classifier and rejection families as well as consistency and calibration results. These theoretical guarantees guide us in designing new algorithms that can exploit different kernel-based hypothesis sets for the classifier and rejection functions. We compare and contrast our general framework with the special case of confidence-based rejection for which we devise alternative loss functions and algorithms as well. We report the results of several experiments showing that our kernel-based algorithms can yield a notable improvement over the best existing confidence-based rejection algorithm.",
author = "Corinna Cortes and Giulia DeSalvo and Mehryar Mohri",
year = "2016",
doi = "10.1007/978-3-319-46379-7_5",
language = "English (US)",
isbn = "9783319463780",
volume = "9925 LNAI",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
publisher = "Springer Verlag",
pages = "67--82",
booktitle = "Algorithmic Learning Theory - 27th International Conference, ALT 2016, Proceedings",
address = "Germany",

}

TY - GEN

T1 - Learning with rejection

AU - Cortes, Corinna

AU - DeSalvo, Giulia

AU - Mohri, Mehryar

PY - 2016

Y1 - 2016

N2 - We introduce a novel framework for classification with a rejection option that consists of simultaneously learning two functions: a classifier along with a rejection function. We present a full theoretical analysis of this framework including new data-dependent learning bounds in terms of the Rademacher complexities of the classifier and rejection families as well as consistency and calibration results. These theoretical guarantees guide us in designing new algorithms that can exploit different kernel-based hypothesis sets for the classifier and rejection functions. We compare and contrast our general framework with the special case of confidence-based rejection for which we devise alternative loss functions and algorithms as well. We report the results of several experiments showing that our kernel-based algorithms can yield a notable improvement over the best existing confidence-based rejection algorithm.

AB - We introduce a novel framework for classification with a rejection option that consists of simultaneously learning two functions: a classifier along with a rejection function. We present a full theoretical analysis of this framework including new data-dependent learning bounds in terms of the Rademacher complexities of the classifier and rejection families as well as consistency and calibration results. These theoretical guarantees guide us in designing new algorithms that can exploit different kernel-based hypothesis sets for the classifier and rejection functions. We compare and contrast our general framework with the special case of confidence-based rejection for which we devise alternative loss functions and algorithms as well. We report the results of several experiments showing that our kernel-based algorithms can yield a notable improvement over the best existing confidence-based rejection algorithm.

UR - http://www.scopus.com/inward/record.url?scp=84994071536&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84994071536&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-46379-7_5

DO - 10.1007/978-3-319-46379-7_5

M3 - Conference contribution

SN - 9783319463780

VL - 9925 LNAI

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 67

EP - 82

BT - Algorithmic Learning Theory - 27th International Conference, ALT 2016, Proceedings

PB - Springer Verlag

ER -