Asymptotic analysis of MAP estimation via the replica method and compressed sensing

Sundeep Rangan, Alyson K. Fletcher, Vivek K. Goyal

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

The replica method is a non-rigorous but widely-accepted technique from statistical physics used in the asymptotic analysis of large, random, nonlinear problems. This paper applies the replica method to non-Gaussian maximum a posteriori (MAP) estimation. It is shown that with random linear measurements and Gaussian noise, the asymptotic behavior of the MAP estimate of an n-dimensional vector "decouples" as n scalar MAP estimators. The result is a counterpart to Guo and Verdú's replica analysis of minimum mean-squared error estimation. The replica MAP analysis can be readily applied to many estimators used in compressed sensing, including basis pursuit, lasso, linear estimation with thresholding, and zero norm-regularized estimation. In the case of lasso estimation the scalar estimator reduces to a soft-thresholding operator, and for zero norm-regularized estimation it reduces to a hard-threshold. Among other benefits, the replica method provides a computationally-tractable method for exactly computing various performance metrics including mean-squared error and sparsity pattern recovery probability.

Original languageEnglish (US)
Title of host publicationAdvances in Neural Information Processing Systems 22 - Proceedings of the 2009 Conference
Pages1545-1553
Number of pages9
StatePublished - 2009
Event23rd Annual Conference on Neural Information Processing Systems, NIPS 2009 - Vancouver, BC, Canada
Duration: Dec 7 2009Dec 10 2009

Other

Other23rd Annual Conference on Neural Information Processing Systems, NIPS 2009
CountryCanada
CityVancouver, BC
Period12/7/0912/10/09

Fingerprint

Compressed sensing
Asymptotic analysis
Error analysis
Mathematical operators
Physics
Recovery

ASJC Scopus subject areas

  • Information Systems

Cite this

Rangan, S., Fletcher, A. K., & Goyal, V. K. (2009). Asymptotic analysis of MAP estimation via the replica method and compressed sensing. In Advances in Neural Information Processing Systems 22 - Proceedings of the 2009 Conference (pp. 1545-1553)

Asymptotic analysis of MAP estimation via the replica method and compressed sensing. / Rangan, Sundeep; Fletcher, Alyson K.; Goyal, Vivek K.

Advances in Neural Information Processing Systems 22 - Proceedings of the 2009 Conference. 2009. p. 1545-1553.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Rangan, S, Fletcher, AK & Goyal, VK 2009, Asymptotic analysis of MAP estimation via the replica method and compressed sensing. in Advances in Neural Information Processing Systems 22 - Proceedings of the 2009 Conference. pp. 1545-1553, 23rd Annual Conference on Neural Information Processing Systems, NIPS 2009, Vancouver, BC, Canada, 12/7/09.
Rangan S, Fletcher AK, Goyal VK. Asymptotic analysis of MAP estimation via the replica method and compressed sensing. In Advances in Neural Information Processing Systems 22 - Proceedings of the 2009 Conference. 2009. p. 1545-1553
Rangan, Sundeep ; Fletcher, Alyson K. ; Goyal, Vivek K. / Asymptotic analysis of MAP estimation via the replica method and compressed sensing. Advances in Neural Information Processing Systems 22 - Proceedings of the 2009 Conference. 2009. pp. 1545-1553
@inproceedings{dc384921b9c947819145a9918af1409d,
title = "Asymptotic analysis of MAP estimation via the replica method and compressed sensing",
abstract = "The replica method is a non-rigorous but widely-accepted technique from statistical physics used in the asymptotic analysis of large, random, nonlinear problems. This paper applies the replica method to non-Gaussian maximum a posteriori (MAP) estimation. It is shown that with random linear measurements and Gaussian noise, the asymptotic behavior of the MAP estimate of an n-dimensional vector {"}decouples{"} as n scalar MAP estimators. The result is a counterpart to Guo and Verd{\'u}'s replica analysis of minimum mean-squared error estimation. The replica MAP analysis can be readily applied to many estimators used in compressed sensing, including basis pursuit, lasso, linear estimation with thresholding, and zero norm-regularized estimation. In the case of lasso estimation the scalar estimator reduces to a soft-thresholding operator, and for zero norm-regularized estimation it reduces to a hard-threshold. Among other benefits, the replica method provides a computationally-tractable method for exactly computing various performance metrics including mean-squared error and sparsity pattern recovery probability.",
author = "Sundeep Rangan and Fletcher, {Alyson K.} and Goyal, {Vivek K.}",
year = "2009",
language = "English (US)",
isbn = "9781615679119",
pages = "1545--1553",
booktitle = "Advances in Neural Information Processing Systems 22 - Proceedings of the 2009 Conference",

}

TY - GEN

T1 - Asymptotic analysis of MAP estimation via the replica method and compressed sensing

AU - Rangan, Sundeep

AU - Fletcher, Alyson K.

AU - Goyal, Vivek K.

PY - 2009

Y1 - 2009

N2 - The replica method is a non-rigorous but widely-accepted technique from statistical physics used in the asymptotic analysis of large, random, nonlinear problems. This paper applies the replica method to non-Gaussian maximum a posteriori (MAP) estimation. It is shown that with random linear measurements and Gaussian noise, the asymptotic behavior of the MAP estimate of an n-dimensional vector "decouples" as n scalar MAP estimators. The result is a counterpart to Guo and Verdú's replica analysis of minimum mean-squared error estimation. The replica MAP analysis can be readily applied to many estimators used in compressed sensing, including basis pursuit, lasso, linear estimation with thresholding, and zero norm-regularized estimation. In the case of lasso estimation the scalar estimator reduces to a soft-thresholding operator, and for zero norm-regularized estimation it reduces to a hard-threshold. Among other benefits, the replica method provides a computationally-tractable method for exactly computing various performance metrics including mean-squared error and sparsity pattern recovery probability.

AB - The replica method is a non-rigorous but widely-accepted technique from statistical physics used in the asymptotic analysis of large, random, nonlinear problems. This paper applies the replica method to non-Gaussian maximum a posteriori (MAP) estimation. It is shown that with random linear measurements and Gaussian noise, the asymptotic behavior of the MAP estimate of an n-dimensional vector "decouples" as n scalar MAP estimators. The result is a counterpart to Guo and Verdú's replica analysis of minimum mean-squared error estimation. The replica MAP analysis can be readily applied to many estimators used in compressed sensing, including basis pursuit, lasso, linear estimation with thresholding, and zero norm-regularized estimation. In the case of lasso estimation the scalar estimator reduces to a soft-thresholding operator, and for zero norm-regularized estimation it reduces to a hard-threshold. Among other benefits, the replica method provides a computationally-tractable method for exactly computing various performance metrics including mean-squared error and sparsity pattern recovery probability.

UR - http://www.scopus.com/inward/record.url?scp=84858722178&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84858722178&partnerID=8YFLogxK

M3 - Conference contribution

SN - 9781615679119

SP - 1545

EP - 1553

BT - Advances in Neural Information Processing Systems 22 - Proceedings of the 2009 Conference

ER -