### Abstract

We present a general study of learning and linear separability with rational kernels, the sequence kernels commonly used in computational biology and natural language processing. We give a characterization of the class of all languages linearly separable with rational kernels and prove several properties of the class of languages linearly separable with a fixed rational kernel. In particular, we show that for kernels with transducer values in a finite set, these languages are necessarily finite Boolean combinations of preimages by a transducer of a single sequence. We also analyze the margin properties of linear separation with rational kernels and show that kernels with transducer values in a finite set guarantee a positive margin and lead to better learning guarantees. Creating a rational kernel with values in a finite set is often non-trivial even for relatively simple cases. However, we present a novel and general algorithm, double-tape disambiguation, that takes as input a transducer mapping sequences to sequence features, and yields an associated transducer that defines a finite range rational kernel. We describe the algorithm in detail and show its application to several cases of interest.

Original language | English (US) |
---|---|

Title of host publication | Learning Theory - 20th Annual Conference on Learning Theory, COLT 2007, Proceedings |

Pages | 349-364 |

Number of pages | 16 |

Volume | 4539 LNAI |

State | Published - 2007 |

Event | 20th Annual Conference on Learning Theory, COLT 2007 - San Diego, CA, United States Duration: Jun 13 2007 → Jun 15 2007 |

### Publication series

Name | Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) |
---|---|

Volume | 4539 LNAI |

ISSN (Print) | 03029743 |

ISSN (Electronic) | 16113349 |

### Other

Other | 20th Annual Conference on Learning Theory, COLT 2007 |
---|---|

Country | United States |

City | San Diego, CA |

Period | 6/13/07 → 6/15/07 |

### Fingerprint

### ASJC Scopus subject areas

- Computer Science(all)
- Biochemistry, Genetics and Molecular Biology(all)
- Theoretical Computer Science

### Cite this

*Learning Theory - 20th Annual Conference on Learning Theory, COLT 2007, Proceedings*(Vol. 4539 LNAI, pp. 349-364). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 4539 LNAI).

**Learning languages with rational kernels.** / Cortes, Corinna; Kontorovich, Leonid; Mohri, Mehryar.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

*Learning Theory - 20th Annual Conference on Learning Theory, COLT 2007, Proceedings.*vol. 4539 LNAI, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 4539 LNAI, pp. 349-364, 20th Annual Conference on Learning Theory, COLT 2007, San Diego, CA, United States, 6/13/07.

}

TY - GEN

T1 - Learning languages with rational kernels

AU - Cortes, Corinna

AU - Kontorovich, Leonid

AU - Mohri, Mehryar

PY - 2007

Y1 - 2007

N2 - We present a general study of learning and linear separability with rational kernels, the sequence kernels commonly used in computational biology and natural language processing. We give a characterization of the class of all languages linearly separable with rational kernels and prove several properties of the class of languages linearly separable with a fixed rational kernel. In particular, we show that for kernels with transducer values in a finite set, these languages are necessarily finite Boolean combinations of preimages by a transducer of a single sequence. We also analyze the margin properties of linear separation with rational kernels and show that kernels with transducer values in a finite set guarantee a positive margin and lead to better learning guarantees. Creating a rational kernel with values in a finite set is often non-trivial even for relatively simple cases. However, we present a novel and general algorithm, double-tape disambiguation, that takes as input a transducer mapping sequences to sequence features, and yields an associated transducer that defines a finite range rational kernel. We describe the algorithm in detail and show its application to several cases of interest.

AB - We present a general study of learning and linear separability with rational kernels, the sequence kernels commonly used in computational biology and natural language processing. We give a characterization of the class of all languages linearly separable with rational kernels and prove several properties of the class of languages linearly separable with a fixed rational kernel. In particular, we show that for kernels with transducer values in a finite set, these languages are necessarily finite Boolean combinations of preimages by a transducer of a single sequence. We also analyze the margin properties of linear separation with rational kernels and show that kernels with transducer values in a finite set guarantee a positive margin and lead to better learning guarantees. Creating a rational kernel with values in a finite set is often non-trivial even for relatively simple cases. However, we present a novel and general algorithm, double-tape disambiguation, that takes as input a transducer mapping sequences to sequence features, and yields an associated transducer that defines a finite range rational kernel. We describe the algorithm in detail and show its application to several cases of interest.

UR - http://www.scopus.com/inward/record.url?scp=38049037764&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=38049037764&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:38049037764

SN - 9783540729259

VL - 4539 LNAI

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 349

EP - 364

BT - Learning Theory - 20th Annual Conference on Learning Theory, COLT 2007, Proceedings

ER -