Sampling techniques for the Nyström method

Sanjiv Kumar, Mehryar Mohri, Ameet Talwalkar

Research output: Contribution to journalArticle

Abstract

The Nyström method is an efficient technique to generate low-rank matrix approximations and is used in several large-scale learning applications. A key aspect of this method is the distribution according to which columns are sampled from the original matrix. In this work, we present an analysis of different sampling techniques for the Nyström method. Our analysis includes both empirical and theoretical components. We first present novel experiments with several real world datasets, comparing the performance of the Nyström method when used with uniform versus non-uniform sampling distributions. Our results suggest that uniform sampling without replacement, in addition to being more efficient both in time and space, produces more effective approximations. This motivates the theoretical part of our analysis which gives the first performance bounds for the Nyström method precisely when used with uniform sampling without replacement.

Original languageEnglish (US)
Pages (from-to)304-311
Number of pages8
JournalJournal of Machine Learning Research
Volume5
StatePublished - 2009

Fingerprint

Sampling
Sampling without Replacement
Nonuniform Sampling
Matrix Approximation
Low-rank Approximation
Low-rank Matrices
Performance Bounds
Sampling Distribution
Approximation
Experiments
Experiment

ASJC Scopus subject areas

  • Artificial Intelligence
  • Software
  • Control and Systems Engineering
  • Statistics and Probability

Cite this

Sampling techniques for the Nyström method. / Kumar, Sanjiv; Mohri, Mehryar; Talwalkar, Ameet.

In: Journal of Machine Learning Research, Vol. 5, 2009, p. 304-311.

Research output: Contribution to journalArticle

Kumar, Sanjiv ; Mohri, Mehryar ; Talwalkar, Ameet. / Sampling techniques for the Nyström method. In: Journal of Machine Learning Research. 2009 ; Vol. 5. pp. 304-311.
@article{4063ca3d61d049a898de177e429fc5e7,
title = "Sampling techniques for the Nystr{\"o}m method",
abstract = "The Nystr{\"o}m method is an efficient technique to generate low-rank matrix approximations and is used in several large-scale learning applications. A key aspect of this method is the distribution according to which columns are sampled from the original matrix. In this work, we present an analysis of different sampling techniques for the Nystr{\"o}m method. Our analysis includes both empirical and theoretical components. We first present novel experiments with several real world datasets, comparing the performance of the Nystr{\"o}m method when used with uniform versus non-uniform sampling distributions. Our results suggest that uniform sampling without replacement, in addition to being more efficient both in time and space, produces more effective approximations. This motivates the theoretical part of our analysis which gives the first performance bounds for the Nystr{\"o}m method precisely when used with uniform sampling without replacement.",
author = "Sanjiv Kumar and Mehryar Mohri and Ameet Talwalkar",
year = "2009",
language = "English (US)",
volume = "5",
pages = "304--311",
journal = "Journal of Machine Learning Research",
issn = "1532-4435",
publisher = "Microtome Publishing",

}

TY - JOUR

T1 - Sampling techniques for the Nyström method

AU - Kumar, Sanjiv

AU - Mohri, Mehryar

AU - Talwalkar, Ameet

PY - 2009

Y1 - 2009

N2 - The Nyström method is an efficient technique to generate low-rank matrix approximations and is used in several large-scale learning applications. A key aspect of this method is the distribution according to which columns are sampled from the original matrix. In this work, we present an analysis of different sampling techniques for the Nyström method. Our analysis includes both empirical and theoretical components. We first present novel experiments with several real world datasets, comparing the performance of the Nyström method when used with uniform versus non-uniform sampling distributions. Our results suggest that uniform sampling without replacement, in addition to being more efficient both in time and space, produces more effective approximations. This motivates the theoretical part of our analysis which gives the first performance bounds for the Nyström method precisely when used with uniform sampling without replacement.

AB - The Nyström method is an efficient technique to generate low-rank matrix approximations and is used in several large-scale learning applications. A key aspect of this method is the distribution according to which columns are sampled from the original matrix. In this work, we present an analysis of different sampling techniques for the Nyström method. Our analysis includes both empirical and theoretical components. We first present novel experiments with several real world datasets, comparing the performance of the Nyström method when used with uniform versus non-uniform sampling distributions. Our results suggest that uniform sampling without replacement, in addition to being more efficient both in time and space, produces more effective approximations. This motivates the theoretical part of our analysis which gives the first performance bounds for the Nyström method precisely when used with uniform sampling without replacement.

UR - http://www.scopus.com/inward/record.url?scp=84862286216&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84862286216&partnerID=8YFLogxK

M3 - Article

VL - 5

SP - 304

EP - 311

JO - Journal of Machine Learning Research

JF - Journal of Machine Learning Research

SN - 1532-4435

ER -