Efficient computation of the relative entropy of probabilistic automata

Corinna Cortes, Mehryar Mohri, Ashish Rastogi, Michael D. Riley

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

The problem of the efficient computation of the relative entropy of two distributions represented by deterministic weighted automata arises in several machine learning problems. We show that this problem can be naturally formulated as a shortest-distance problem over an intersection automaton denned on an appropriate semiring. We describe simple and efficient novel algorithms for its computation and report the results of experiments demonstrating the practicality of our algorithms for very large weighted automata. Our algorithms apply to unambiguous weighted automata, a class of weighted automata that strictly includes deterministic weighted automata. These are also the first algorithms extending the computation of entropy or of relative entropy beyond the class of deterministic weighted automata.

Original languageEnglish (US)
Title of host publicationLATIN 2006: Theoretical Informatics - 7th Latin American Symposium, Proceedings
Pages323-336
Number of pages14
Volume3887 LNCS
DOIs
StatePublished - 2006
EventLATIN 2006: Theoretical Informatics - 7th Latin American Symposium - Valdivia, Chile
Duration: Mar 20 2006Mar 24 2006

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume3887 LNCS
ISSN (Print)03029743
ISSN (Electronic)16113349

Other

OtherLATIN 2006: Theoretical Informatics - 7th Latin American Symposium
CountryChile
CityValdivia
Period3/20/063/24/06

Fingerprint

Weighted Automata
Probabilistic Automata
Relative Entropy
Entropy
Learning systems
Semiring
Automata
Machine Learning
Efficient Algorithms
Strictly
Intersection
Experiments
Experiment

ASJC Scopus subject areas

  • Computer Science(all)
  • Biochemistry, Genetics and Molecular Biology(all)
  • Theoretical Computer Science

Cite this

Cortes, C., Mohri, M., Rastogi, A., & Riley, M. D. (2006). Efficient computation of the relative entropy of probabilistic automata. In LATIN 2006: Theoretical Informatics - 7th Latin American Symposium, Proceedings (Vol. 3887 LNCS, pp. 323-336). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 3887 LNCS). https://doi.org/10.1007/11682462_32

Efficient computation of the relative entropy of probabilistic automata. / Cortes, Corinna; Mohri, Mehryar; Rastogi, Ashish; Riley, Michael D.

LATIN 2006: Theoretical Informatics - 7th Latin American Symposium, Proceedings. Vol. 3887 LNCS 2006. p. 323-336 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 3887 LNCS).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Cortes, C, Mohri, M, Rastogi, A & Riley, MD 2006, Efficient computation of the relative entropy of probabilistic automata. in LATIN 2006: Theoretical Informatics - 7th Latin American Symposium, Proceedings. vol. 3887 LNCS, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 3887 LNCS, pp. 323-336, LATIN 2006: Theoretical Informatics - 7th Latin American Symposium, Valdivia, Chile, 3/20/06. https://doi.org/10.1007/11682462_32
Cortes C, Mohri M, Rastogi A, Riley MD. Efficient computation of the relative entropy of probabilistic automata. In LATIN 2006: Theoretical Informatics - 7th Latin American Symposium, Proceedings. Vol. 3887 LNCS. 2006. p. 323-336. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/11682462_32
Cortes, Corinna ; Mohri, Mehryar ; Rastogi, Ashish ; Riley, Michael D. / Efficient computation of the relative entropy of probabilistic automata. LATIN 2006: Theoretical Informatics - 7th Latin American Symposium, Proceedings. Vol. 3887 LNCS 2006. pp. 323-336 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{97fdbb238fc4462bb1072592ccc128e4,
title = "Efficient computation of the relative entropy of probabilistic automata",
abstract = "The problem of the efficient computation of the relative entropy of two distributions represented by deterministic weighted automata arises in several machine learning problems. We show that this problem can be naturally formulated as a shortest-distance problem over an intersection automaton denned on an appropriate semiring. We describe simple and efficient novel algorithms for its computation and report the results of experiments demonstrating the practicality of our algorithms for very large weighted automata. Our algorithms apply to unambiguous weighted automata, a class of weighted automata that strictly includes deterministic weighted automata. These are also the first algorithms extending the computation of entropy or of relative entropy beyond the class of deterministic weighted automata.",
author = "Corinna Cortes and Mehryar Mohri and Ashish Rastogi and Riley, {Michael D.}",
year = "2006",
doi = "10.1007/11682462_32",
language = "English (US)",
isbn = "354032755X",
volume = "3887 LNCS",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
pages = "323--336",
booktitle = "LATIN 2006: Theoretical Informatics - 7th Latin American Symposium, Proceedings",

}

TY - GEN

T1 - Efficient computation of the relative entropy of probabilistic automata

AU - Cortes, Corinna

AU - Mohri, Mehryar

AU - Rastogi, Ashish

AU - Riley, Michael D.

PY - 2006

Y1 - 2006

N2 - The problem of the efficient computation of the relative entropy of two distributions represented by deterministic weighted automata arises in several machine learning problems. We show that this problem can be naturally formulated as a shortest-distance problem over an intersection automaton denned on an appropriate semiring. We describe simple and efficient novel algorithms for its computation and report the results of experiments demonstrating the practicality of our algorithms for very large weighted automata. Our algorithms apply to unambiguous weighted automata, a class of weighted automata that strictly includes deterministic weighted automata. These are also the first algorithms extending the computation of entropy or of relative entropy beyond the class of deterministic weighted automata.

AB - The problem of the efficient computation of the relative entropy of two distributions represented by deterministic weighted automata arises in several machine learning problems. We show that this problem can be naturally formulated as a shortest-distance problem over an intersection automaton denned on an appropriate semiring. We describe simple and efficient novel algorithms for its computation and report the results of experiments demonstrating the practicality of our algorithms for very large weighted automata. Our algorithms apply to unambiguous weighted automata, a class of weighted automata that strictly includes deterministic weighted automata. These are also the first algorithms extending the computation of entropy or of relative entropy beyond the class of deterministic weighted automata.

UR - http://www.scopus.com/inward/record.url?scp=33745625833&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=33745625833&partnerID=8YFLogxK

U2 - 10.1007/11682462_32

DO - 10.1007/11682462_32

M3 - Conference contribution

SN - 354032755X

SN - 9783540327554

VL - 3887 LNCS

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 323

EP - 336

BT - LATIN 2006: Theoretical Informatics - 7th Latin American Symposium, Proceedings

ER -