A dual coordinate descent algorithm for SVMs combined with rational kernels

Cyril Allauzen, Corinna Cortes, Mehryar Mohri

Research output: Contribution to journalArticle

Abstract

This paper presents a novel application of automata algorithms to machine learning. It introduces the first optimization solution for support vector machines used with sequence kernels that is purely based on weighted automata and transducer algorithms, without requiring any specific solver. The algorithms presented apply to a family of kernels covering all those commonly used in text and speech processing or computational biology. We show that these algorithms have significantly better computational complexity than previous ones and report the results of large-scale experiments demonstrating a dramatic reduction of the training time, typically by several orders of magnitude.

Original languageEnglish (US)
Pages (from-to)1761-1779
Number of pages19
JournalInternational Journal of Foundations of Computer Science
Volume22
Issue number8
DOIs
StatePublished - Dec 2011

Fingerprint

Text processing
Speech processing
Support vector machines
Learning systems
Transducers
Computational complexity
Experiments

Keywords

  • coordinate descent
  • Machine learning
  • optimization
  • rational kernels
  • support vector machines
  • weighted automata

ASJC Scopus subject areas

  • Computer Science (miscellaneous)

Cite this

A dual coordinate descent algorithm for SVMs combined with rational kernels. / Allauzen, Cyril; Cortes, Corinna; Mohri, Mehryar.

In: International Journal of Foundations of Computer Science, Vol. 22, No. 8, 12.2011, p. 1761-1779.

Research output: Contribution to journalArticle

@article{3d12c8f104704a0cbc016f074f43d688,
title = "A dual coordinate descent algorithm for SVMs combined with rational kernels",
abstract = "This paper presents a novel application of automata algorithms to machine learning. It introduces the first optimization solution for support vector machines used with sequence kernels that is purely based on weighted automata and transducer algorithms, without requiring any specific solver. The algorithms presented apply to a family of kernels covering all those commonly used in text and speech processing or computational biology. We show that these algorithms have significantly better computational complexity than previous ones and report the results of large-scale experiments demonstrating a dramatic reduction of the training time, typically by several orders of magnitude.",
keywords = "coordinate descent, Machine learning, optimization, rational kernels, support vector machines, weighted automata",
author = "Cyril Allauzen and Corinna Cortes and Mehryar Mohri",
year = "2011",
month = "12",
doi = "10.1142/S0129054111009021",
language = "English (US)",
volume = "22",
pages = "1761--1779",
journal = "International Journal of Foundations of Computer Science",
issn = "0129-0541",
publisher = "World Scientific Publishing Co. Pte Ltd",
number = "8",

}

TY - JOUR

T1 - A dual coordinate descent algorithm for SVMs combined with rational kernels

AU - Allauzen, Cyril

AU - Cortes, Corinna

AU - Mohri, Mehryar

PY - 2011/12

Y1 - 2011/12

N2 - This paper presents a novel application of automata algorithms to machine learning. It introduces the first optimization solution for support vector machines used with sequence kernels that is purely based on weighted automata and transducer algorithms, without requiring any specific solver. The algorithms presented apply to a family of kernels covering all those commonly used in text and speech processing or computational biology. We show that these algorithms have significantly better computational complexity than previous ones and report the results of large-scale experiments demonstrating a dramatic reduction of the training time, typically by several orders of magnitude.

AB - This paper presents a novel application of automata algorithms to machine learning. It introduces the first optimization solution for support vector machines used with sequence kernels that is purely based on weighted automata and transducer algorithms, without requiring any specific solver. The algorithms presented apply to a family of kernels covering all those commonly used in text and speech processing or computational biology. We show that these algorithms have significantly better computational complexity than previous ones and report the results of large-scale experiments demonstrating a dramatic reduction of the training time, typically by several orders of magnitude.

KW - coordinate descent

KW - Machine learning

KW - optimization

KW - rational kernels

KW - support vector machines

KW - weighted automata

UR - http://www.scopus.com/inward/record.url?scp=84855757917&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84855757917&partnerID=8YFLogxK

U2 - 10.1142/S0129054111009021

DO - 10.1142/S0129054111009021

M3 - Article

VL - 22

SP - 1761

EP - 1779

JO - International Journal of Foundations of Computer Science

JF - International Journal of Foundations of Computer Science

SN - 0129-0541

IS - 8

ER -