Generalization bounds for time series prediction with non-stationary processes

Vitaly Kuznetsov, Mehryar Mohri

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

This paper presents the first generalization bounds for time series prediction with a non-stationary mixing stochastic process. We prove Rademacher complexity learning bounds for both averagepath generalization with non-stationary β-mixing processes and pathdependent generalization with non-stationary φ-mixing processes. Our guarantees are expressed in terms of β- or φ-mixing coefficients and a natural measure of discrepancy between training and target distributions. They admit as special cases previous Rademacher complexity bounds for non-i.i.d. stationary distributions, for independent but not identically distributed random variables, or for the i.i.d. case. We show that, using a new sub-sample selection technique we introduce, our bounds can be tightened under the natural assumption of convergent stochastic processes. We also prove that fast learning rates can be achieved by extending existing local Rademacher complexity analysis to non-i.i.d. setting.

Original languageEnglish (US)
Title of host publicationAlgorithmic Learning Theory - 25th International Conference, ALT 2014, Proceedings
EditorsAlexander Clark, Thomas Zeugmann, Sandra Zilles, Peter Auer
PublisherSpringer Verlag
Pages260-274
Number of pages15
ISBN (Electronic)9783319116617
StatePublished - Jan 1 2014
Event25th International Conference on Algorithmic Learning Theory, ALT 2014 - Bled, Slovenia
Duration: Oct 8 2014Oct 10 2014

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume8776
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

Other25th International Conference on Algorithmic Learning Theory, ALT 2014
CountrySlovenia
CityBled
Period10/8/1410/10/14

    Fingerprint

Keywords

  • Fast rates
  • Generalization bounds
  • Local Rademacher complexity
  • Mixing
  • Stationary processes
  • Time series

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Cite this

Kuznetsov, V., & Mohri, M. (2014). Generalization bounds for time series prediction with non-stationary processes. In A. Clark, T. Zeugmann, S. Zilles, & P. Auer (Eds.), Algorithmic Learning Theory - 25th International Conference, ALT 2014, Proceedings (pp. 260-274). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 8776). Springer Verlag.