Learning languages with rational kernels

Corinna Cortes, Leonid Kontorovich, Mehryar Mohri

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We present a general study of learning and linear separability with rational kernels, the sequence kernels commonly used in computational biology and natural language processing. We give a characterization of the class of all languages linearly separable with rational kernels and prove several properties of the class of languages linearly separable with a fixed rational kernel. In particular, we show that for kernels with transducer values in a finite set, these languages are necessarily finite Boolean combinations of preimages by a transducer of a single sequence. We also analyze the margin properties of linear separation with rational kernels and show that kernels with transducer values in a finite set guarantee a positive margin and lead to better learning guarantees. Creating a rational kernel with values in a finite set is often non-trivial even for relatively simple cases. However, we present a novel and general algorithm, double-tape disambiguation, that takes as input a transducer mapping sequences to sequence features, and yields an associated transducer that defines a finite range rational kernel. We describe the algorithm in detail and show its application to several cases of interest.

Original languageEnglish (US)
Title of host publicationLearning Theory - 20th Annual Conference on Learning Theory, COLT 2007, Proceedings
Pages349-364
Number of pages16
Volume4539 LNAI
StatePublished - 2007
Event20th Annual Conference on Learning Theory, COLT 2007 - San Diego, CA, United States
Duration: Jun 13 2007Jun 15 2007

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume4539 LNAI
ISSN (Print)03029743
ISSN (Electronic)16113349

Other

Other20th Annual Conference on Learning Theory, COLT 2007
CountryUnited States
CitySan Diego, CA
Period6/13/076/15/07

Fingerprint

Transducers
Language
Learning
kernel
Transducer
Natural Language Processing
Finite Set
Margin
Computational Biology
Tapes
Linearly
Language Acquisition
Separability
Natural Language
Processing
Range of data

ASJC Scopus subject areas

  • Computer Science(all)
  • Biochemistry, Genetics and Molecular Biology(all)
  • Theoretical Computer Science

Cite this

Cortes, C., Kontorovich, L., & Mohri, M. (2007). Learning languages with rational kernels. In Learning Theory - 20th Annual Conference on Learning Theory, COLT 2007, Proceedings (Vol. 4539 LNAI, pp. 349-364). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 4539 LNAI).

Learning languages with rational kernels. / Cortes, Corinna; Kontorovich, Leonid; Mohri, Mehryar.

Learning Theory - 20th Annual Conference on Learning Theory, COLT 2007, Proceedings. Vol. 4539 LNAI 2007. p. 349-364 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 4539 LNAI).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Cortes, C, Kontorovich, L & Mohri, M 2007, Learning languages with rational kernels. in Learning Theory - 20th Annual Conference on Learning Theory, COLT 2007, Proceedings. vol. 4539 LNAI, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 4539 LNAI, pp. 349-364, 20th Annual Conference on Learning Theory, COLT 2007, San Diego, CA, United States, 6/13/07.
Cortes C, Kontorovich L, Mohri M. Learning languages with rational kernels. In Learning Theory - 20th Annual Conference on Learning Theory, COLT 2007, Proceedings. Vol. 4539 LNAI. 2007. p. 349-364. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
Cortes, Corinna ; Kontorovich, Leonid ; Mohri, Mehryar. / Learning languages with rational kernels. Learning Theory - 20th Annual Conference on Learning Theory, COLT 2007, Proceedings. Vol. 4539 LNAI 2007. pp. 349-364 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{78c6651687f345f89935261561c4218d,
title = "Learning languages with rational kernels",
abstract = "We present a general study of learning and linear separability with rational kernels, the sequence kernels commonly used in computational biology and natural language processing. We give a characterization of the class of all languages linearly separable with rational kernels and prove several properties of the class of languages linearly separable with a fixed rational kernel. In particular, we show that for kernels with transducer values in a finite set, these languages are necessarily finite Boolean combinations of preimages by a transducer of a single sequence. We also analyze the margin properties of linear separation with rational kernels and show that kernels with transducer values in a finite set guarantee a positive margin and lead to better learning guarantees. Creating a rational kernel with values in a finite set is often non-trivial even for relatively simple cases. However, we present a novel and general algorithm, double-tape disambiguation, that takes as input a transducer mapping sequences to sequence features, and yields an associated transducer that defines a finite range rational kernel. We describe the algorithm in detail and show its application to several cases of interest.",
author = "Corinna Cortes and Leonid Kontorovich and Mehryar Mohri",
year = "2007",
language = "English (US)",
isbn = "9783540729259",
volume = "4539 LNAI",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
pages = "349--364",
booktitle = "Learning Theory - 20th Annual Conference on Learning Theory, COLT 2007, Proceedings",

}

TY - GEN

T1 - Learning languages with rational kernels

AU - Cortes, Corinna

AU - Kontorovich, Leonid

AU - Mohri, Mehryar

PY - 2007

Y1 - 2007

N2 - We present a general study of learning and linear separability with rational kernels, the sequence kernels commonly used in computational biology and natural language processing. We give a characterization of the class of all languages linearly separable with rational kernels and prove several properties of the class of languages linearly separable with a fixed rational kernel. In particular, we show that for kernels with transducer values in a finite set, these languages are necessarily finite Boolean combinations of preimages by a transducer of a single sequence. We also analyze the margin properties of linear separation with rational kernels and show that kernels with transducer values in a finite set guarantee a positive margin and lead to better learning guarantees. Creating a rational kernel with values in a finite set is often non-trivial even for relatively simple cases. However, we present a novel and general algorithm, double-tape disambiguation, that takes as input a transducer mapping sequences to sequence features, and yields an associated transducer that defines a finite range rational kernel. We describe the algorithm in detail and show its application to several cases of interest.

AB - We present a general study of learning and linear separability with rational kernels, the sequence kernels commonly used in computational biology and natural language processing. We give a characterization of the class of all languages linearly separable with rational kernels and prove several properties of the class of languages linearly separable with a fixed rational kernel. In particular, we show that for kernels with transducer values in a finite set, these languages are necessarily finite Boolean combinations of preimages by a transducer of a single sequence. We also analyze the margin properties of linear separation with rational kernels and show that kernels with transducer values in a finite set guarantee a positive margin and lead to better learning guarantees. Creating a rational kernel with values in a finite set is often non-trivial even for relatively simple cases. However, we present a novel and general algorithm, double-tape disambiguation, that takes as input a transducer mapping sequences to sequence features, and yields an associated transducer that defines a finite range rational kernel. We describe the algorithm in detail and show its application to several cases of interest.

UR - http://www.scopus.com/inward/record.url?scp=38049037764&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=38049037764&partnerID=8YFLogxK

M3 - Conference contribution

SN - 9783540729259

VL - 4539 LNAI

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 349

EP - 364

BT - Learning Theory - 20th Annual Conference on Learning Theory, COLT 2007, Proceedings

ER -