The Landscape of the Spiked Tensor Model

Gerard Ben Arous, Song Mei, Andrea Montanari, Mihai Nica

Research output: Contribution to journalArticle

Abstract

We consider the problem of estimating a large rank-one tensor u⊗k ∈ (ℝn)⊗k, k ≥ 3, in Gaussian noise. Earlier work characterized a critical signal-to-noise ratio λBayes = O(1) above which an ideal estimator achieves strictly positive correlation with the unknown vector of interest. Remarkably, no polynomial-time algorithm is known that achieved this goal unless λ ≥ Cn(k − 2)/4, and even powerful semidefinite programming relaxations appear to fail for 1 ≪ λ ≪ n(k − 2)/4. In order to elucidate this behavior, we consider the maximum likelihood estimator, which requires maximizing a degree-k homogeneous polynomial over the unit sphere in n dimensions. We compute the expected number of critical points and local maxima of this objective function and show that it is exponential in the dimensions n, and give exact formulas for the exponential growth rate. We show that (for λ larger than a constant) critical points are either very close to the unknown vector u or are confined in a band of width Θ(λ−1/(k − 1)) around the maximum circle that is orthogonal to u. For local maxima, this band shrinks to be of size Θ(λ−1/(k − 2)). These “uninformative” local maxima are likely to cause the failure of optimization algorithms.

Original languageEnglish (US)
JournalCommunications on Pure and Applied Mathematics
DOIs
StateAccepted/In press - Jan 1 2019

Fingerprint

Tensors
Tensor
Polynomials
Maximum likelihood
Critical point
Signal to noise ratio
Semidefinite Programming Relaxation
Unknown
Homogeneous Polynomials
Gaussian Noise
Strictly positive
Exponential Growth
Bayes
Unit Sphere
Model
Maximum Likelihood Estimator
Polynomial-time Algorithm
Optimization Algorithm
Circle
Objective function

ASJC Scopus subject areas

  • Mathematics(all)
  • Applied Mathematics

Cite this

The Landscape of the Spiked Tensor Model. / Ben Arous, Gerard; Mei, Song; Montanari, Andrea; Nica, Mihai.

In: Communications on Pure and Applied Mathematics, 01.01.2019.

Research output: Contribution to journalArticle

@article{718580a5082c48cb96101b3ae8ae7f0d,
title = "The Landscape of the Spiked Tensor Model",
abstract = "We consider the problem of estimating a large rank-one tensor u⊗k ∈ (ℝn)⊗k, k ≥ 3, in Gaussian noise. Earlier work characterized a critical signal-to-noise ratio λBayes = O(1) above which an ideal estimator achieves strictly positive correlation with the unknown vector of interest. Remarkably, no polynomial-time algorithm is known that achieved this goal unless λ ≥ Cn(k − 2)/4, and even powerful semidefinite programming relaxations appear to fail for 1 ≪ λ ≪ n(k − 2)/4. In order to elucidate this behavior, we consider the maximum likelihood estimator, which requires maximizing a degree-k homogeneous polynomial over the unit sphere in n dimensions. We compute the expected number of critical points and local maxima of this objective function and show that it is exponential in the dimensions n, and give exact formulas for the exponential growth rate. We show that (for λ larger than a constant) critical points are either very close to the unknown vector u or are confined in a band of width Θ(λ−1/(k − 1)) around the maximum circle that is orthogonal to u. For local maxima, this band shrinks to be of size Θ(λ−1/(k − 2)). These “uninformative” local maxima are likely to cause the failure of optimization algorithms.",
author = "{Ben Arous}, Gerard and Song Mei and Andrea Montanari and Mihai Nica",
year = "2019",
month = "1",
day = "1",
doi = "10.1002/cpa.21861",
language = "English (US)",
journal = "Communications on Pure and Applied Mathematics",
issn = "0010-3640",
publisher = "Wiley-Liss Inc.",

}

TY - JOUR

T1 - The Landscape of the Spiked Tensor Model

AU - Ben Arous, Gerard

AU - Mei, Song

AU - Montanari, Andrea

AU - Nica, Mihai

PY - 2019/1/1

Y1 - 2019/1/1

N2 - We consider the problem of estimating a large rank-one tensor u⊗k ∈ (ℝn)⊗k, k ≥ 3, in Gaussian noise. Earlier work characterized a critical signal-to-noise ratio λBayes = O(1) above which an ideal estimator achieves strictly positive correlation with the unknown vector of interest. Remarkably, no polynomial-time algorithm is known that achieved this goal unless λ ≥ Cn(k − 2)/4, and even powerful semidefinite programming relaxations appear to fail for 1 ≪ λ ≪ n(k − 2)/4. In order to elucidate this behavior, we consider the maximum likelihood estimator, which requires maximizing a degree-k homogeneous polynomial over the unit sphere in n dimensions. We compute the expected number of critical points and local maxima of this objective function and show that it is exponential in the dimensions n, and give exact formulas for the exponential growth rate. We show that (for λ larger than a constant) critical points are either very close to the unknown vector u or are confined in a band of width Θ(λ−1/(k − 1)) around the maximum circle that is orthogonal to u. For local maxima, this band shrinks to be of size Θ(λ−1/(k − 2)). These “uninformative” local maxima are likely to cause the failure of optimization algorithms.

AB - We consider the problem of estimating a large rank-one tensor u⊗k ∈ (ℝn)⊗k, k ≥ 3, in Gaussian noise. Earlier work characterized a critical signal-to-noise ratio λBayes = O(1) above which an ideal estimator achieves strictly positive correlation with the unknown vector of interest. Remarkably, no polynomial-time algorithm is known that achieved this goal unless λ ≥ Cn(k − 2)/4, and even powerful semidefinite programming relaxations appear to fail for 1 ≪ λ ≪ n(k − 2)/4. In order to elucidate this behavior, we consider the maximum likelihood estimator, which requires maximizing a degree-k homogeneous polynomial over the unit sphere in n dimensions. We compute the expected number of critical points and local maxima of this objective function and show that it is exponential in the dimensions n, and give exact formulas for the exponential growth rate. We show that (for λ larger than a constant) critical points are either very close to the unknown vector u or are confined in a band of width Θ(λ−1/(k − 1)) around the maximum circle that is orthogonal to u. For local maxima, this band shrinks to be of size Θ(λ−1/(k − 2)). These “uninformative” local maxima are likely to cause the failure of optimization algorithms.

UR - http://www.scopus.com/inward/record.url?scp=85070771190&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85070771190&partnerID=8YFLogxK

U2 - 10.1002/cpa.21861

DO - 10.1002/cpa.21861

M3 - Article

AN - SCOPUS:85070771190

JO - Communications on Pure and Applied Mathematics

JF - Communications on Pure and Applied Mathematics

SN - 0010-3640

ER -