Rademacher complexity bounds for non-i.i.d. processes

Mehryar Mohri, Afshin Rostamizadeh

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

This paper presents the first Rademacher complexity-based error bounds for noni. i.d. settings, a generalization of similar existing bounds derived for the i.i.d. case. Our bounds hold in the scenario of dependent samples generated by a stationary β-mixing process, which is commonly adopted in many previous studies of noni. i.d. settings. They benefit fromthe crucial advantages of Rademacher complexity over other measures of the complexity of hypothesis classes. In particular, they are data-dependent and measure the complexity of a class of hypotheses based on the training sample. The empirical Rademacher complexity can be estimated from such finite samples and lead to tighter generalization bounds. We also present the first margin bounds for kernel-based classification in this non-i.i.d. setting and briefly study their convergence.

Original languageEnglish (US)
Title of host publicationAdvances in Neural Information Processing Systems 21 - Proceedings of the 2008 Conference
Pages1097-1104
Number of pages8
StatePublished - 2009
Event22nd Annual Conference on Neural Information Processing Systems, NIPS 2008 - Vancouver, BC, Canada
Duration: Dec 8 2008Dec 11 2008

Other

Other22nd Annual Conference on Neural Information Processing Systems, NIPS 2008
CountryCanada
CityVancouver, BC
Period12/8/0812/11/08

ASJC Scopus subject areas

  • Information Systems

Cite this

Mohri, M., & Rostamizadeh, A. (2009). Rademacher complexity bounds for non-i.i.d. processes. In Advances in Neural Information Processing Systems 21 - Proceedings of the 2008 Conference (pp. 1097-1104)

Rademacher complexity bounds for non-i.i.d. processes. / Mohri, Mehryar; Rostamizadeh, Afshin.

Advances in Neural Information Processing Systems 21 - Proceedings of the 2008 Conference. 2009. p. 1097-1104.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Mohri, M & Rostamizadeh, A 2009, Rademacher complexity bounds for non-i.i.d. processes. in Advances in Neural Information Processing Systems 21 - Proceedings of the 2008 Conference. pp. 1097-1104, 22nd Annual Conference on Neural Information Processing Systems, NIPS 2008, Vancouver, BC, Canada, 12/8/08.
Mohri M, Rostamizadeh A. Rademacher complexity bounds for non-i.i.d. processes. In Advances in Neural Information Processing Systems 21 - Proceedings of the 2008 Conference. 2009. p. 1097-1104
Mohri, Mehryar ; Rostamizadeh, Afshin. / Rademacher complexity bounds for non-i.i.d. processes. Advances in Neural Information Processing Systems 21 - Proceedings of the 2008 Conference. 2009. pp. 1097-1104
@inproceedings{ea845b9605004b6781ee9211dc21d86d,
title = "Rademacher complexity bounds for non-i.i.d. processes",
abstract = "This paper presents the first Rademacher complexity-based error bounds for noni. i.d. settings, a generalization of similar existing bounds derived for the i.i.d. case. Our bounds hold in the scenario of dependent samples generated by a stationary β-mixing process, which is commonly adopted in many previous studies of noni. i.d. settings. They benefit fromthe crucial advantages of Rademacher complexity over other measures of the complexity of hypothesis classes. In particular, they are data-dependent and measure the complexity of a class of hypotheses based on the training sample. The empirical Rademacher complexity can be estimated from such finite samples and lead to tighter generalization bounds. We also present the first margin bounds for kernel-based classification in this non-i.i.d. setting and briefly study their convergence.",
author = "Mehryar Mohri and Afshin Rostamizadeh",
year = "2009",
language = "English (US)",
isbn = "9781605609492",
pages = "1097--1104",
booktitle = "Advances in Neural Information Processing Systems 21 - Proceedings of the 2008 Conference",

}

TY - GEN

T1 - Rademacher complexity bounds for non-i.i.d. processes

AU - Mohri, Mehryar

AU - Rostamizadeh, Afshin

PY - 2009

Y1 - 2009

N2 - This paper presents the first Rademacher complexity-based error bounds for noni. i.d. settings, a generalization of similar existing bounds derived for the i.i.d. case. Our bounds hold in the scenario of dependent samples generated by a stationary β-mixing process, which is commonly adopted in many previous studies of noni. i.d. settings. They benefit fromthe crucial advantages of Rademacher complexity over other measures of the complexity of hypothesis classes. In particular, they are data-dependent and measure the complexity of a class of hypotheses based on the training sample. The empirical Rademacher complexity can be estimated from such finite samples and lead to tighter generalization bounds. We also present the first margin bounds for kernel-based classification in this non-i.i.d. setting and briefly study their convergence.

AB - This paper presents the first Rademacher complexity-based error bounds for noni. i.d. settings, a generalization of similar existing bounds derived for the i.i.d. case. Our bounds hold in the scenario of dependent samples generated by a stationary β-mixing process, which is commonly adopted in many previous studies of noni. i.d. settings. They benefit fromthe crucial advantages of Rademacher complexity over other measures of the complexity of hypothesis classes. In particular, they are data-dependent and measure the complexity of a class of hypotheses based on the training sample. The empirical Rademacher complexity can be estimated from such finite samples and lead to tighter generalization bounds. We also present the first margin bounds for kernel-based classification in this non-i.i.d. setting and briefly study their convergence.

UR - http://www.scopus.com/inward/record.url?scp=84858764746&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84858764746&partnerID=8YFLogxK

M3 - Conference contribution

SN - 9781605609492

SP - 1097

EP - 1104

BT - Advances in Neural Information Processing Systems 21 - Proceedings of the 2008 Conference

ER -