Ensemble nyström

Sanjiv Kumar, Mehryar Mohri, Ameet Talwalkar

Research output: Chapter in Book/Report/Conference proceedingChapter

Abstract

A crucial technique for scaling kernel methods to very large datasets reaching or exceeding millions of instances is based on low-rank approximation of kernel matrices. The Nyström method is a popular technique to generate low-rank matrix approximations but it requires sampling of a large number of columns from the original matrix to achieve good accuracy. This chapter describes a new family of algorithms based on mixtures of Nyström approximations, Ensemble Nyström algorithms, that yield more accurate low-rank approximations than the standard Nyström method. We give a detailed study of variants of these algorithms based on simple averaging, an exponential weight method, and regression-based methods. A theoretical analysis of these algorithms, including novel error bounds guaranteeing a better convergence rate than the standard Nyström method is also presented. Finally, experiments with several datasets containing up to 1 M points are presented, demonstrating significant improvement over the standard Nyström approximation.

Original languageEnglish (US)
Title of host publicationEnsemble Machine Learning: Methods and Applications
PublisherSpringer US
Pages203-223
Number of pages21
ISBN (Print)9781441993267, 9781441993250
DOIs
StatePublished - Jan 1 2012

Fingerprint

Sampling
Experiments

ASJC Scopus subject areas

  • Computer Science(all)

Cite this

Kumar, S., Mohri, M., & Talwalkar, A. (2012). Ensemble nyström. In Ensemble Machine Learning: Methods and Applications (pp. 203-223). Springer US. https://doi.org/10.1007/9781441993267_7

Ensemble nyström. / Kumar, Sanjiv; Mohri, Mehryar; Talwalkar, Ameet.

Ensemble Machine Learning: Methods and Applications. Springer US, 2012. p. 203-223.

Research output: Chapter in Book/Report/Conference proceedingChapter

Kumar, S, Mohri, M & Talwalkar, A 2012, Ensemble nyström. in Ensemble Machine Learning: Methods and Applications. Springer US, pp. 203-223. https://doi.org/10.1007/9781441993267_7
Kumar S, Mohri M, Talwalkar A. Ensemble nyström. In Ensemble Machine Learning: Methods and Applications. Springer US. 2012. p. 203-223 https://doi.org/10.1007/9781441993267_7
Kumar, Sanjiv ; Mohri, Mehryar ; Talwalkar, Ameet. / Ensemble nyström. Ensemble Machine Learning: Methods and Applications. Springer US, 2012. pp. 203-223
@inbook{036027b514c246b7b85f2cb8ed8068e8,
title = "Ensemble nystr{\"o}m",
abstract = "A crucial technique for scaling kernel methods to very large datasets reaching or exceeding millions of instances is based on low-rank approximation of kernel matrices. The Nystr{\"o}m method is a popular technique to generate low-rank matrix approximations but it requires sampling of a large number of columns from the original matrix to achieve good accuracy. This chapter describes a new family of algorithms based on mixtures of Nystr{\"o}m approximations, Ensemble Nystr{\"o}m algorithms, that yield more accurate low-rank approximations than the standard Nystr{\"o}m method. We give a detailed study of variants of these algorithms based on simple averaging, an exponential weight method, and regression-based methods. A theoretical analysis of these algorithms, including novel error bounds guaranteeing a better convergence rate than the standard Nystr{\"o}m method is also presented. Finally, experiments with several datasets containing up to 1 M points are presented, demonstrating significant improvement over the standard Nystr{\"o}m approximation.",
author = "Sanjiv Kumar and Mehryar Mohri and Ameet Talwalkar",
year = "2012",
month = "1",
day = "1",
doi = "10.1007/9781441993267_7",
language = "English (US)",
isbn = "9781441993267",
pages = "203--223",
booktitle = "Ensemble Machine Learning: Methods and Applications",
publisher = "Springer US",

}

TY - CHAP

T1 - Ensemble nyström

AU - Kumar, Sanjiv

AU - Mohri, Mehryar

AU - Talwalkar, Ameet

PY - 2012/1/1

Y1 - 2012/1/1

N2 - A crucial technique for scaling kernel methods to very large datasets reaching or exceeding millions of instances is based on low-rank approximation of kernel matrices. The Nyström method is a popular technique to generate low-rank matrix approximations but it requires sampling of a large number of columns from the original matrix to achieve good accuracy. This chapter describes a new family of algorithms based on mixtures of Nyström approximations, Ensemble Nyström algorithms, that yield more accurate low-rank approximations than the standard Nyström method. We give a detailed study of variants of these algorithms based on simple averaging, an exponential weight method, and regression-based methods. A theoretical analysis of these algorithms, including novel error bounds guaranteeing a better convergence rate than the standard Nyström method is also presented. Finally, experiments with several datasets containing up to 1 M points are presented, demonstrating significant improvement over the standard Nyström approximation.

AB - A crucial technique for scaling kernel methods to very large datasets reaching or exceeding millions of instances is based on low-rank approximation of kernel matrices. The Nyström method is a popular technique to generate low-rank matrix approximations but it requires sampling of a large number of columns from the original matrix to achieve good accuracy. This chapter describes a new family of algorithms based on mixtures of Nyström approximations, Ensemble Nyström algorithms, that yield more accurate low-rank approximations than the standard Nyström method. We give a detailed study of variants of these algorithms based on simple averaging, an exponential weight method, and regression-based methods. A theoretical analysis of these algorithms, including novel error bounds guaranteeing a better convergence rate than the standard Nyström method is also presented. Finally, experiments with several datasets containing up to 1 M points are presented, demonstrating significant improvement over the standard Nyström approximation.

UR - http://www.scopus.com/inward/record.url?scp=84955352335&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84955352335&partnerID=8YFLogxK

U2 - 10.1007/9781441993267_7

DO - 10.1007/9781441993267_7

M3 - Chapter

SN - 9781441993267

SN - 9781441993250

SP - 203

EP - 223

BT - Ensemble Machine Learning: Methods and Applications

PB - Springer US

ER -