Sampling methods for the Nyström method

Sanjiv Kumar, Mehryar Mohri, Ameet Talwalkar

Research output: Contribution to journalArticle

Abstract

The Nyström method is an efficient technique to generate low-rank matrix approximations and is used in several large-scale learning applications. A key aspect of this method is the procedure according to which columns are sampled from the original matrix. In this work, we explore the efficacy of a variety of fixed and adaptive sampling schemes. We also propose a family of ensemble- based sampling algorithms for the Nyström method. We report results of extensive experiments that provide a detailed comparison of various fixed and adaptive sampling techniques, and demonstrate the performance improvement associated with the ensemble Nyström method when used in conjunction with either fixed or adaptive sampling schemes. Corroborating these empirical findings, we present a theoretical analysis of the Nyström method, providing novel error bounds guaranteeing a better convergence rate of the ensemble Nyström method in comparison to the standard Nyström method.

Original languageEnglish (US)
Pages (from-to)981-1006
Number of pages26
JournalJournal of Machine Learning Research
Volume13
StatePublished - Apr 2012

Fingerprint

Sampling Methods
Adaptive Sampling
Sampling
Ensemble Methods
Matrix Approximation
Low-rank Approximation
Low-rank Matrices
Error Bounds
Efficacy
Theoretical Analysis
Rate of Convergence
Ensemble
Experiments
Demonstrate
Experiment

Keywords

  • Ensemble methods
  • Large-scale learning
  • Low-rank approximation
  • Nyström method

ASJC Scopus subject areas

  • Artificial Intelligence
  • Software
  • Control and Systems Engineering
  • Statistics and Probability

Cite this

Sampling methods for the Nyström method. / Kumar, Sanjiv; Mohri, Mehryar; Talwalkar, Ameet.

In: Journal of Machine Learning Research, Vol. 13, 04.2012, p. 981-1006.

Research output: Contribution to journalArticle

Kumar, S, Mohri, M & Talwalkar, A 2012, 'Sampling methods for the Nyström method', Journal of Machine Learning Research, vol. 13, pp. 981-1006.
Kumar, Sanjiv ; Mohri, Mehryar ; Talwalkar, Ameet. / Sampling methods for the Nyström method. In: Journal of Machine Learning Research. 2012 ; Vol. 13. pp. 981-1006.
@article{3c30eeaf855f4c1fb7c2100eff3a77eb,
title = "Sampling methods for the Nystr{\"o}m method",
abstract = "The Nystr{\"o}m method is an efficient technique to generate low-rank matrix approximations and is used in several large-scale learning applications. A key aspect of this method is the procedure according to which columns are sampled from the original matrix. In this work, we explore the efficacy of a variety of fixed and adaptive sampling schemes. We also propose a family of ensemble- based sampling algorithms for the Nystr{\"o}m method. We report results of extensive experiments that provide a detailed comparison of various fixed and adaptive sampling techniques, and demonstrate the performance improvement associated with the ensemble Nystr{\"o}m method when used in conjunction with either fixed or adaptive sampling schemes. Corroborating these empirical findings, we present a theoretical analysis of the Nystr{\"o}m method, providing novel error bounds guaranteeing a better convergence rate of the ensemble Nystr{\"o}m method in comparison to the standard Nystr{\"o}m method.",
keywords = "Ensemble methods, Large-scale learning, Low-rank approximation, Nystr{\"o}m method",
author = "Sanjiv Kumar and Mehryar Mohri and Ameet Talwalkar",
year = "2012",
month = "4",
language = "English (US)",
volume = "13",
pages = "981--1006",
journal = "Journal of Machine Learning Research",
issn = "1532-4435",
publisher = "Microtome Publishing",

}

TY - JOUR

T1 - Sampling methods for the Nyström method

AU - Kumar, Sanjiv

AU - Mohri, Mehryar

AU - Talwalkar, Ameet

PY - 2012/4

Y1 - 2012/4

N2 - The Nyström method is an efficient technique to generate low-rank matrix approximations and is used in several large-scale learning applications. A key aspect of this method is the procedure according to which columns are sampled from the original matrix. In this work, we explore the efficacy of a variety of fixed and adaptive sampling schemes. We also propose a family of ensemble- based sampling algorithms for the Nyström method. We report results of extensive experiments that provide a detailed comparison of various fixed and adaptive sampling techniques, and demonstrate the performance improvement associated with the ensemble Nyström method when used in conjunction with either fixed or adaptive sampling schemes. Corroborating these empirical findings, we present a theoretical analysis of the Nyström method, providing novel error bounds guaranteeing a better convergence rate of the ensemble Nyström method in comparison to the standard Nyström method.

AB - The Nyström method is an efficient technique to generate low-rank matrix approximations and is used in several large-scale learning applications. A key aspect of this method is the procedure according to which columns are sampled from the original matrix. In this work, we explore the efficacy of a variety of fixed and adaptive sampling schemes. We also propose a family of ensemble- based sampling algorithms for the Nyström method. We report results of extensive experiments that provide a detailed comparison of various fixed and adaptive sampling techniques, and demonstrate the performance improvement associated with the ensemble Nyström method when used in conjunction with either fixed or adaptive sampling schemes. Corroborating these empirical findings, we present a theoretical analysis of the Nyström method, providing novel error bounds guaranteeing a better convergence rate of the ensemble Nyström method in comparison to the standard Nyström method.

KW - Ensemble methods

KW - Large-scale learning

KW - Low-rank approximation

KW - Nyström method

UR - http://www.scopus.com/inward/record.url?scp=84860672647&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84860672647&partnerID=8YFLogxK

M3 - Article

VL - 13

SP - 981

EP - 1006

JO - Journal of Machine Learning Research

JF - Journal of Machine Learning Research

SN - 1532-4435

ER -