Optimality and sub-optimality of PCA I: Spiked random matrix models

Amelia Perry, Alexander S. Wein, Afonso Bandeira, Ankur Moitra

Research output: Contribution to journalArticle

Abstract

A central problem of random matrix theory is to understand the eigenvalues of spiked random matrix models, introduced by Johnstone, in which a prominent eigenvector (or “spike”) is planted into a random matrix. These distributions form natural statistical models for principal component analysis (PCA) problems throughout the sciences. Baik, Ben Arous and Péché showed that the spiked Wishart ensemble exhibits a sharp phase transition asymptotically: when the spike strength is above a critical threshold, it is possible to detect the presence of a spike based on the top eigenvalue, and below the threshold the top eigenvalue provides no information. Such results form the basis of our understanding of when PCA can detect a low-rank signal in the presence of noise. However, under structural assumptions on the spike, not all information is necessarily contained in the spectrum. We study the statistical limits of tests for the presence of a spike, including nonspectral tests. Our results leverage Le Cam's notion of contiguity and include: (i) For the Gaussian Wigner ensemble, we show that PCA achieves the optimal detection threshold for certain natural priors for the spike. (ii) For any non-Gaussian Wigner ensemble, PCA is sub-optimal for detection. However, an efficient variant of PCA achieves the optimal threshold (for natural priors) by pre-transforming the matrix entries. (iii) For the Gaussian Wishart ensemble, the PCA threshold is optimal for positive spikes (for natural priors) but this is not always the case for negative spikes.

Original languageEnglish (US)
Pages (from-to)2416-2451
Number of pages36
JournalAnnals of Statistics
Volume46
Issue number5
DOIs
StatePublished - Oct 1 2018

Fingerprint

Matrix Models
Random Matrices
Spike
Principal Component Analysis
Optimality
Ensemble
Eigenvalue
Contiguity
Critical Threshold
Principal component analysis
Random Matrix Theory
Leverage
Eigenvector
Statistical Model
Phase Transition
Eigenvalues

Keywords

  • Contiguity
  • Deformed Wigner
  • Hypothesis testing
  • Phase transition
  • Power envelope
  • Principal component analysis
  • Random matrix
  • Spiked covariance

ASJC Scopus subject areas

  • Statistics and Probability
  • Statistics, Probability and Uncertainty

Cite this

Perry, A., Wein, A. S., Bandeira, A., & Moitra, A. (2018). Optimality and sub-optimality of PCA I: Spiked random matrix models. Annals of Statistics, 46(5), 2416-2451. https://doi.org/10.1214/17-AOS1625

Optimality and sub-optimality of PCA I : Spiked random matrix models. / Perry, Amelia; Wein, Alexander S.; Bandeira, Afonso; Moitra, Ankur.

In: Annals of Statistics, Vol. 46, No. 5, 01.10.2018, p. 2416-2451.

Research output: Contribution to journalArticle

Perry, A, Wein, AS, Bandeira, A & Moitra, A 2018, 'Optimality and sub-optimality of PCA I: Spiked random matrix models', Annals of Statistics, vol. 46, no. 5, pp. 2416-2451. https://doi.org/10.1214/17-AOS1625
Perry, Amelia ; Wein, Alexander S. ; Bandeira, Afonso ; Moitra, Ankur. / Optimality and sub-optimality of PCA I : Spiked random matrix models. In: Annals of Statistics. 2018 ; Vol. 46, No. 5. pp. 2416-2451.
@article{381123c1a9c14240a170913476768a8e,
title = "Optimality and sub-optimality of PCA I: Spiked random matrix models",
abstract = "A central problem of random matrix theory is to understand the eigenvalues of spiked random matrix models, introduced by Johnstone, in which a prominent eigenvector (or “spike”) is planted into a random matrix. These distributions form natural statistical models for principal component analysis (PCA) problems throughout the sciences. Baik, Ben Arous and P{\'e}ch{\'e} showed that the spiked Wishart ensemble exhibits a sharp phase transition asymptotically: when the spike strength is above a critical threshold, it is possible to detect the presence of a spike based on the top eigenvalue, and below the threshold the top eigenvalue provides no information. Such results form the basis of our understanding of when PCA can detect a low-rank signal in the presence of noise. However, under structural assumptions on the spike, not all information is necessarily contained in the spectrum. We study the statistical limits of tests for the presence of a spike, including nonspectral tests. Our results leverage Le Cam's notion of contiguity and include: (i) For the Gaussian Wigner ensemble, we show that PCA achieves the optimal detection threshold for certain natural priors for the spike. (ii) For any non-Gaussian Wigner ensemble, PCA is sub-optimal for detection. However, an efficient variant of PCA achieves the optimal threshold (for natural priors) by pre-transforming the matrix entries. (iii) For the Gaussian Wishart ensemble, the PCA threshold is optimal for positive spikes (for natural priors) but this is not always the case for negative spikes.",
keywords = "Contiguity, Deformed Wigner, Hypothesis testing, Phase transition, Power envelope, Principal component analysis, Random matrix, Spiked covariance",
author = "Amelia Perry and Wein, {Alexander S.} and Afonso Bandeira and Ankur Moitra",
year = "2018",
month = "10",
day = "1",
doi = "10.1214/17-AOS1625",
language = "English (US)",
volume = "46",
pages = "2416--2451",
journal = "Annals of Statistics",
issn = "0090-5364",
publisher = "Institute of Mathematical Statistics",
number = "5",

}

TY - JOUR

T1 - Optimality and sub-optimality of PCA I

T2 - Spiked random matrix models

AU - Perry, Amelia

AU - Wein, Alexander S.

AU - Bandeira, Afonso

AU - Moitra, Ankur

PY - 2018/10/1

Y1 - 2018/10/1

N2 - A central problem of random matrix theory is to understand the eigenvalues of spiked random matrix models, introduced by Johnstone, in which a prominent eigenvector (or “spike”) is planted into a random matrix. These distributions form natural statistical models for principal component analysis (PCA) problems throughout the sciences. Baik, Ben Arous and Péché showed that the spiked Wishart ensemble exhibits a sharp phase transition asymptotically: when the spike strength is above a critical threshold, it is possible to detect the presence of a spike based on the top eigenvalue, and below the threshold the top eigenvalue provides no information. Such results form the basis of our understanding of when PCA can detect a low-rank signal in the presence of noise. However, under structural assumptions on the spike, not all information is necessarily contained in the spectrum. We study the statistical limits of tests for the presence of a spike, including nonspectral tests. Our results leverage Le Cam's notion of contiguity and include: (i) For the Gaussian Wigner ensemble, we show that PCA achieves the optimal detection threshold for certain natural priors for the spike. (ii) For any non-Gaussian Wigner ensemble, PCA is sub-optimal for detection. However, an efficient variant of PCA achieves the optimal threshold (for natural priors) by pre-transforming the matrix entries. (iii) For the Gaussian Wishart ensemble, the PCA threshold is optimal for positive spikes (for natural priors) but this is not always the case for negative spikes.

AB - A central problem of random matrix theory is to understand the eigenvalues of spiked random matrix models, introduced by Johnstone, in which a prominent eigenvector (or “spike”) is planted into a random matrix. These distributions form natural statistical models for principal component analysis (PCA) problems throughout the sciences. Baik, Ben Arous and Péché showed that the spiked Wishart ensemble exhibits a sharp phase transition asymptotically: when the spike strength is above a critical threshold, it is possible to detect the presence of a spike based on the top eigenvalue, and below the threshold the top eigenvalue provides no information. Such results form the basis of our understanding of when PCA can detect a low-rank signal in the presence of noise. However, under structural assumptions on the spike, not all information is necessarily contained in the spectrum. We study the statistical limits of tests for the presence of a spike, including nonspectral tests. Our results leverage Le Cam's notion of contiguity and include: (i) For the Gaussian Wigner ensemble, we show that PCA achieves the optimal detection threshold for certain natural priors for the spike. (ii) For any non-Gaussian Wigner ensemble, PCA is sub-optimal for detection. However, an efficient variant of PCA achieves the optimal threshold (for natural priors) by pre-transforming the matrix entries. (iii) For the Gaussian Wishart ensemble, the PCA threshold is optimal for positive spikes (for natural priors) but this is not always the case for negative spikes.

KW - Contiguity

KW - Deformed Wigner

KW - Hypothesis testing

KW - Phase transition

KW - Power envelope

KW - Principal component analysis

KW - Random matrix

KW - Spiked covariance

UR - http://www.scopus.com/inward/record.url?scp=85052637597&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85052637597&partnerID=8YFLogxK

U2 - 10.1214/17-AOS1625

DO - 10.1214/17-AOS1625

M3 - Article

AN - SCOPUS:85052637597

VL - 46

SP - 2416

EP - 2451

JO - Annals of Statistics

JF - Annals of Statistics

SN - 0090-5364

IS - 5

ER -