Structural online learning

Mehryar Mohri, Scott Yang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We study the problem of learning ensembles in the online setting, when the hypotheses are selected out of a base family that may be a union of possibly very complex sub-families. We prove new theoretical guarantees for the online learning of such ensembles in terms of the sequential Rademacher complexities of these sub-families. We also describe an algorithm that benefits from such guarantees. We further extend our framework by proving new structural estimation error guarantees for ensembles in the batch setting through a new data-dependent online-to-batch conversion technique, thereby also devising an effective algorithm for the batch setting which does not require the estimation of the Rademacher complexities of base sub-families.

Original languageEnglish (US)
Title of host publicationAlgorithmic Learning Theory - 27th International Conference, ALT 2016, Proceedings
PublisherSpringer Verlag
Pages223-237
Number of pages15
Volume9925 LNAI
ISBN (Print)9783319463780
DOIs
StatePublished - 2016
Event27th International Conference on Algorithmic Learning Theory, ALT 2016 - Bari, Italy
Duration: Oct 19 2016Oct 21 2016

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume9925 LNAI
ISSN (Print)03029743
ISSN (Electronic)16113349

Other

Other27th International Conference on Algorithmic Learning Theory, ALT 2016
CountryItaly
CityBari
Period10/19/1610/21/16

Fingerprint

Online Learning
Batch
Error analysis
Ensemble
Ensemble Learning
Dependent Data
Estimation Error
Union
Family

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Cite this

Mohri, M., & Yang, S. (2016). Structural online learning. In Algorithmic Learning Theory - 27th International Conference, ALT 2016, Proceedings (Vol. 9925 LNAI, pp. 223-237). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9925 LNAI). Springer Verlag. https://doi.org/10.1007/978-3-319-46379-7_15

Structural online learning. / Mohri, Mehryar; Yang, Scott.

Algorithmic Learning Theory - 27th International Conference, ALT 2016, Proceedings. Vol. 9925 LNAI Springer Verlag, 2016. p. 223-237 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9925 LNAI).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Mohri, M & Yang, S 2016, Structural online learning. in Algorithmic Learning Theory - 27th International Conference, ALT 2016, Proceedings. vol. 9925 LNAI, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 9925 LNAI, Springer Verlag, pp. 223-237, 27th International Conference on Algorithmic Learning Theory, ALT 2016, Bari, Italy, 10/19/16. https://doi.org/10.1007/978-3-319-46379-7_15
Mohri M, Yang S. Structural online learning. In Algorithmic Learning Theory - 27th International Conference, ALT 2016, Proceedings. Vol. 9925 LNAI. Springer Verlag. 2016. p. 223-237. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-319-46379-7_15
Mohri, Mehryar ; Yang, Scott. / Structural online learning. Algorithmic Learning Theory - 27th International Conference, ALT 2016, Proceedings. Vol. 9925 LNAI Springer Verlag, 2016. pp. 223-237 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{32314ab71793405585324ea59c30b234,
title = "Structural online learning",
abstract = "We study the problem of learning ensembles in the online setting, when the hypotheses are selected out of a base family that may be a union of possibly very complex sub-families. We prove new theoretical guarantees for the online learning of such ensembles in terms of the sequential Rademacher complexities of these sub-families. We also describe an algorithm that benefits from such guarantees. We further extend our framework by proving new structural estimation error guarantees for ensembles in the batch setting through a new data-dependent online-to-batch conversion technique, thereby also devising an effective algorithm for the batch setting which does not require the estimation of the Rademacher complexities of base sub-families.",
author = "Mehryar Mohri and Scott Yang",
year = "2016",
doi = "10.1007/978-3-319-46379-7_15",
language = "English (US)",
isbn = "9783319463780",
volume = "9925 LNAI",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
publisher = "Springer Verlag",
pages = "223--237",
booktitle = "Algorithmic Learning Theory - 27th International Conference, ALT 2016, Proceedings",
address = "Germany",

}

TY - GEN

T1 - Structural online learning

AU - Mohri, Mehryar

AU - Yang, Scott

PY - 2016

Y1 - 2016

N2 - We study the problem of learning ensembles in the online setting, when the hypotheses are selected out of a base family that may be a union of possibly very complex sub-families. We prove new theoretical guarantees for the online learning of such ensembles in terms of the sequential Rademacher complexities of these sub-families. We also describe an algorithm that benefits from such guarantees. We further extend our framework by proving new structural estimation error guarantees for ensembles in the batch setting through a new data-dependent online-to-batch conversion technique, thereby also devising an effective algorithm for the batch setting which does not require the estimation of the Rademacher complexities of base sub-families.

AB - We study the problem of learning ensembles in the online setting, when the hypotheses are selected out of a base family that may be a union of possibly very complex sub-families. We prove new theoretical guarantees for the online learning of such ensembles in terms of the sequential Rademacher complexities of these sub-families. We also describe an algorithm that benefits from such guarantees. We further extend our framework by proving new structural estimation error guarantees for ensembles in the batch setting through a new data-dependent online-to-batch conversion technique, thereby also devising an effective algorithm for the batch setting which does not require the estimation of the Rademacher complexities of base sub-families.

UR - http://www.scopus.com/inward/record.url?scp=84994076380&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84994076380&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-46379-7_15

DO - 10.1007/978-3-319-46379-7_15

M3 - Conference contribution

AN - SCOPUS:84994076380

SN - 9783319463780

VL - 9925 LNAI

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 223

EP - 237

BT - Algorithmic Learning Theory - 27th International Conference, ALT 2016, Proceedings

PB - Springer Verlag

ER -