Asymptotic analysis of MAP estimation via the replica method and applications to compressed sensing

Sundeep Rangan, Alyson K. Fletcher, Vivek K. Goyal

Research output: Contribution to journalArticle

Abstract

The replica method is a nonrigorous but well-known technique from statistical physics used in the asymptotic analysis of large, random, nonlinear problems. This paper applies the replica method, under the assumption of replica symmetry, to study estimators that are maximum a posteriori (MAP) under a postulated prior distribution. It is shown that with random linear measurements and Gaussian noise, the replica-symmetric prediction of the asymptotic behavior of the postulated MAP estimate of an n-dimensional vector "decouples" as n scalar postulated MAP estimators. The result is based on applying a hardening argument to the replica analysis of postulated posterior mean estimators of Tanaka and of Guo and Verd. The replica-symmetric postulated MAP analysis can be readily applied to many estimators used in compressed sensing, including basis pursuit, least absolute shrinkage and selection operator (LASSO), linear estimation with thresholding, and zero norm-regularized estimation. In the case of LASSO estimation, the scalar estimator reduces to a soft-thresholding operator, and for zero norm-regularized estimation, it reduces to a hard threshold. Among other benefits, the replica method provides a computationally tractable method for precisely predicting various performance metrics including mean-squared error and sparsity pattern recovery probability.

Original languageEnglish (US)
Article number6157073
Pages (from-to)1902-1923
Number of pages22
JournalIEEE Transactions on Information Theory
Volume58
Issue number3
DOIs
StatePublished - Mar 2012

Fingerprint

Compressed sensing
Asymptotic analysis
Mathematical operators
Hardening
physics
Physics
Recovery
performance

Keywords

  • Compressed sensing
  • Laplace's method
  • large deviations
  • least absolute shrinkage and selection operator (LASSO)
  • non-Gaussian estimation
  • nonlinear estimation
  • random matrices
  • sparsity
  • spin glasses
  • statistical mechanics
  • thresholding

ASJC Scopus subject areas

  • Information Systems
  • Computer Science Applications
  • Library and Information Sciences

Cite this

Asymptotic analysis of MAP estimation via the replica method and applications to compressed sensing. / Rangan, Sundeep; Fletcher, Alyson K.; Goyal, Vivek K.

In: IEEE Transactions on Information Theory, Vol. 58, No. 3, 6157073, 03.2012, p. 1902-1923.

Research output: Contribution to journalArticle

@article{2f126c190b4c4c55897f3cabef046612,
title = "Asymptotic analysis of MAP estimation via the replica method and applications to compressed sensing",
abstract = "The replica method is a nonrigorous but well-known technique from statistical physics used in the asymptotic analysis of large, random, nonlinear problems. This paper applies the replica method, under the assumption of replica symmetry, to study estimators that are maximum a posteriori (MAP) under a postulated prior distribution. It is shown that with random linear measurements and Gaussian noise, the replica-symmetric prediction of the asymptotic behavior of the postulated MAP estimate of an n-dimensional vector {"}decouples{"} as n scalar postulated MAP estimators. The result is based on applying a hardening argument to the replica analysis of postulated posterior mean estimators of Tanaka and of Guo and Verd. The replica-symmetric postulated MAP analysis can be readily applied to many estimators used in compressed sensing, including basis pursuit, least absolute shrinkage and selection operator (LASSO), linear estimation with thresholding, and zero norm-regularized estimation. In the case of LASSO estimation, the scalar estimator reduces to a soft-thresholding operator, and for zero norm-regularized estimation, it reduces to a hard threshold. Among other benefits, the replica method provides a computationally tractable method for precisely predicting various performance metrics including mean-squared error and sparsity pattern recovery probability.",
keywords = "Compressed sensing, Laplace's method, large deviations, least absolute shrinkage and selection operator (LASSO), non-Gaussian estimation, nonlinear estimation, random matrices, sparsity, spin glasses, statistical mechanics, thresholding",
author = "Sundeep Rangan and Fletcher, {Alyson K.} and Goyal, {Vivek K.}",
year = "2012",
month = "3",
doi = "10.1109/TIT.2011.2177575",
language = "English (US)",
volume = "58",
pages = "1902--1923",
journal = "IEEE Transactions on Information Theory",
issn = "0018-9448",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
number = "3",

}

TY - JOUR

T1 - Asymptotic analysis of MAP estimation via the replica method and applications to compressed sensing

AU - Rangan, Sundeep

AU - Fletcher, Alyson K.

AU - Goyal, Vivek K.

PY - 2012/3

Y1 - 2012/3

N2 - The replica method is a nonrigorous but well-known technique from statistical physics used in the asymptotic analysis of large, random, nonlinear problems. This paper applies the replica method, under the assumption of replica symmetry, to study estimators that are maximum a posteriori (MAP) under a postulated prior distribution. It is shown that with random linear measurements and Gaussian noise, the replica-symmetric prediction of the asymptotic behavior of the postulated MAP estimate of an n-dimensional vector "decouples" as n scalar postulated MAP estimators. The result is based on applying a hardening argument to the replica analysis of postulated posterior mean estimators of Tanaka and of Guo and Verd. The replica-symmetric postulated MAP analysis can be readily applied to many estimators used in compressed sensing, including basis pursuit, least absolute shrinkage and selection operator (LASSO), linear estimation with thresholding, and zero norm-regularized estimation. In the case of LASSO estimation, the scalar estimator reduces to a soft-thresholding operator, and for zero norm-regularized estimation, it reduces to a hard threshold. Among other benefits, the replica method provides a computationally tractable method for precisely predicting various performance metrics including mean-squared error and sparsity pattern recovery probability.

AB - The replica method is a nonrigorous but well-known technique from statistical physics used in the asymptotic analysis of large, random, nonlinear problems. This paper applies the replica method, under the assumption of replica symmetry, to study estimators that are maximum a posteriori (MAP) under a postulated prior distribution. It is shown that with random linear measurements and Gaussian noise, the replica-symmetric prediction of the asymptotic behavior of the postulated MAP estimate of an n-dimensional vector "decouples" as n scalar postulated MAP estimators. The result is based on applying a hardening argument to the replica analysis of postulated posterior mean estimators of Tanaka and of Guo and Verd. The replica-symmetric postulated MAP analysis can be readily applied to many estimators used in compressed sensing, including basis pursuit, least absolute shrinkage and selection operator (LASSO), linear estimation with thresholding, and zero norm-regularized estimation. In the case of LASSO estimation, the scalar estimator reduces to a soft-thresholding operator, and for zero norm-regularized estimation, it reduces to a hard threshold. Among other benefits, the replica method provides a computationally tractable method for precisely predicting various performance metrics including mean-squared error and sparsity pattern recovery probability.

KW - Compressed sensing

KW - Laplace's method

KW - large deviations

KW - least absolute shrinkage and selection operator (LASSO)

KW - non-Gaussian estimation

KW - nonlinear estimation

KW - random matrices

KW - sparsity

KW - spin glasses

KW - statistical mechanics

KW - thresholding

UR - http://www.scopus.com/inward/record.url?scp=84857715972&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84857715972&partnerID=8YFLogxK

U2 - 10.1109/TIT.2011.2177575

DO - 10.1109/TIT.2011.2177575

M3 - Article

AN - SCOPUS:84857715972

VL - 58

SP - 1902

EP - 1923

JO - IEEE Transactions on Information Theory

JF - IEEE Transactions on Information Theory

SN - 0018-9448

IS - 3

M1 - 6157073

ER -