On the rademacher complexity of weighted automata

Borja Balle, Mehryar Mohri

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Weighted automata (WFAs) provide a general framework for the representation of functions mapping strings to real numbers. They include as special instances deterministic finite automata (DFAs), hidden Markov models (HMMs), and predictive states representations (PSRs). In recent years, there has been a renewed interest in weighted automata in machine learning due to the development of efficient and provably correct spectral algorithms for learning weighted automata. Despite the effectiveness reported for spectral techniques in real-world problems, almost all existing statistical guarantees for spectral learning of weighted automata rely on a strong realizability assumption. In this paper, we initiate a systematic study of the learning guarantees for broad classes of weighted automata in an agnostic setting. Our results include bounds on the Rademacher complexity of three general classes of weighted automata, each described in terms of different natural quantities. Interestingly, these bounds underline the key role of different data-dependent parameters in the convergence rates.

Original languageEnglish (US)
Title of host publicationAlgorithmic Learning Theory - 26th International Conference, ALT 2015
PublisherSpringer Verlag
Pages179-193
Number of pages15
Volume9355
ISBN (Print)9783319244853
DOIs
StatePublished - 2015
Event26th International Conference on Algorithmic Learning Theory, ALT 2015 - Banff, Canada
Duration: Oct 4 2015Oct 6 2015

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume9355
ISSN (Print)03029743
ISSN (Electronic)16113349

Other

Other26th International Conference on Algorithmic Learning Theory, ALT 2015
CountryCanada
CityBanff
Period10/4/1510/6/15

Fingerprint

Weighted Automata
Finite automata
Hidden Markov models
Learning systems
Learning Automata
Deterministic Finite Automata
Realizability
Dependent Data
Markov Model
Convergence Rate
Machine Learning
Strings

ASJC Scopus subject areas

  • Computer Science(all)
  • Theoretical Computer Science

Cite this

Balle, B., & Mohri, M. (2015). On the rademacher complexity of weighted automata. In Algorithmic Learning Theory - 26th International Conference, ALT 2015 (Vol. 9355, pp. 179-193). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9355). Springer Verlag. https://doi.org/10.1007/978-3-319-24486-0_12

On the rademacher complexity of weighted automata. / Balle, Borja; Mohri, Mehryar.

Algorithmic Learning Theory - 26th International Conference, ALT 2015. Vol. 9355 Springer Verlag, 2015. p. 179-193 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9355).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Balle, B & Mohri, M 2015, On the rademacher complexity of weighted automata. in Algorithmic Learning Theory - 26th International Conference, ALT 2015. vol. 9355, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 9355, Springer Verlag, pp. 179-193, 26th International Conference on Algorithmic Learning Theory, ALT 2015, Banff, Canada, 10/4/15. https://doi.org/10.1007/978-3-319-24486-0_12
Balle B, Mohri M. On the rademacher complexity of weighted automata. In Algorithmic Learning Theory - 26th International Conference, ALT 2015. Vol. 9355. Springer Verlag. 2015. p. 179-193. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-319-24486-0_12
Balle, Borja ; Mohri, Mehryar. / On the rademacher complexity of weighted automata. Algorithmic Learning Theory - 26th International Conference, ALT 2015. Vol. 9355 Springer Verlag, 2015. pp. 179-193 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{97dffcbd5e924ff0b02da8462b133dc1,
title = "On the rademacher complexity of weighted automata",
abstract = "Weighted automata (WFAs) provide a general framework for the representation of functions mapping strings to real numbers. They include as special instances deterministic finite automata (DFAs), hidden Markov models (HMMs), and predictive states representations (PSRs). In recent years, there has been a renewed interest in weighted automata in machine learning due to the development of efficient and provably correct spectral algorithms for learning weighted automata. Despite the effectiveness reported for spectral techniques in real-world problems, almost all existing statistical guarantees for spectral learning of weighted automata rely on a strong realizability assumption. In this paper, we initiate a systematic study of the learning guarantees for broad classes of weighted automata in an agnostic setting. Our results include bounds on the Rademacher complexity of three general classes of weighted automata, each described in terms of different natural quantities. Interestingly, these bounds underline the key role of different data-dependent parameters in the convergence rates.",
author = "Borja Balle and Mehryar Mohri",
year = "2015",
doi = "10.1007/978-3-319-24486-0_12",
language = "English (US)",
isbn = "9783319244853",
volume = "9355",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
publisher = "Springer Verlag",
pages = "179--193",
booktitle = "Algorithmic Learning Theory - 26th International Conference, ALT 2015",

}

TY - GEN

T1 - On the rademacher complexity of weighted automata

AU - Balle, Borja

AU - Mohri, Mehryar

PY - 2015

Y1 - 2015

N2 - Weighted automata (WFAs) provide a general framework for the representation of functions mapping strings to real numbers. They include as special instances deterministic finite automata (DFAs), hidden Markov models (HMMs), and predictive states representations (PSRs). In recent years, there has been a renewed interest in weighted automata in machine learning due to the development of efficient and provably correct spectral algorithms for learning weighted automata. Despite the effectiveness reported for spectral techniques in real-world problems, almost all existing statistical guarantees for spectral learning of weighted automata rely on a strong realizability assumption. In this paper, we initiate a systematic study of the learning guarantees for broad classes of weighted automata in an agnostic setting. Our results include bounds on the Rademacher complexity of three general classes of weighted automata, each described in terms of different natural quantities. Interestingly, these bounds underline the key role of different data-dependent parameters in the convergence rates.

AB - Weighted automata (WFAs) provide a general framework for the representation of functions mapping strings to real numbers. They include as special instances deterministic finite automata (DFAs), hidden Markov models (HMMs), and predictive states representations (PSRs). In recent years, there has been a renewed interest in weighted automata in machine learning due to the development of efficient and provably correct spectral algorithms for learning weighted automata. Despite the effectiveness reported for spectral techniques in real-world problems, almost all existing statistical guarantees for spectral learning of weighted automata rely on a strong realizability assumption. In this paper, we initiate a systematic study of the learning guarantees for broad classes of weighted automata in an agnostic setting. Our results include bounds on the Rademacher complexity of three general classes of weighted automata, each described in terms of different natural quantities. Interestingly, these bounds underline the key role of different data-dependent parameters in the convergence rates.

UR - http://www.scopus.com/inward/record.url?scp=84945932358&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84945932358&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-24486-0_12

DO - 10.1007/978-3-319-24486-0_12

M3 - Conference contribution

SN - 9783319244853

VL - 9355

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 179

EP - 193

BT - Algorithmic Learning Theory - 26th International Conference, ALT 2015

PB - Springer Verlag

ER -