On the convergence of approximate message passing with arbitrary matrices

Sundeep Rangan, Philip Schniter, Alyson Fletcher

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Approximate message passing (AMP) methods and their variants have attracted considerable recent attention for the problem of estimating a random vector x observed through a linear transform A. In the case of large i.i.d. A, the methods exhibit fast convergence with precise analytic characterizations on the algorithm behavior. However, the convergence of AMP under general transforms is not fully understood. In this paper, we provide sufficient conditions for the convergence of a damped version of the generalized AMP (GAMP) algorithm in the case of Gaussian distributions. It is shown that, with sufficient damping the algorithm can be guaranteed to converge, but the amount of damping grows with peak-to-average ratio of the squared singular values of A. This condition explains the good performance of AMP methods on i.i.d. matrices, but also their difficulties with other classes of transforms. A related sufficient condition is then derived for the local stability of the damped GAMP method under more general (possibly non-Gaussian) distributions, assuming certain strict convexity conditions.

Original languageEnglish (US)
Title of host publication2014 IEEE International Symposium on Information Theory, ISIT 2014
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages236-240
Number of pages5
ISBN (Print)9781479951864
DOIs
StatePublished - 2014
Event2014 IEEE International Symposium on Information Theory, ISIT 2014 - Honolulu, HI, United States
Duration: Jun 29 2014Jul 4 2014

Other

Other2014 IEEE International Symposium on Information Theory, ISIT 2014
CountryUnited States
CityHonolulu, HI
Period6/29/147/4/14

Fingerprint

Message passing
Message Passing
Arbitrary
Transform
Damped
Damping
Strict Convexity
Message-passing Algorithms
Approximate Algorithm
Sufficient Conditions
Gaussian distribution
Local Stability
Singular Values
Random Vector
Sufficient
Converge

Keywords

  • Belief propagation
  • message passing
  • primal-dual methods

ASJC Scopus subject areas

  • Applied Mathematics
  • Modeling and Simulation
  • Theoretical Computer Science
  • Information Systems

Cite this

Rangan, S., Schniter, P., & Fletcher, A. (2014). On the convergence of approximate message passing with arbitrary matrices. In 2014 IEEE International Symposium on Information Theory, ISIT 2014 (pp. 236-240). [6874830] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ISIT.2014.6874830

On the convergence of approximate message passing with arbitrary matrices. / Rangan, Sundeep; Schniter, Philip; Fletcher, Alyson.

2014 IEEE International Symposium on Information Theory, ISIT 2014. Institute of Electrical and Electronics Engineers Inc., 2014. p. 236-240 6874830.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Rangan, S, Schniter, P & Fletcher, A 2014, On the convergence of approximate message passing with arbitrary matrices. in 2014 IEEE International Symposium on Information Theory, ISIT 2014., 6874830, Institute of Electrical and Electronics Engineers Inc., pp. 236-240, 2014 IEEE International Symposium on Information Theory, ISIT 2014, Honolulu, HI, United States, 6/29/14. https://doi.org/10.1109/ISIT.2014.6874830
Rangan S, Schniter P, Fletcher A. On the convergence of approximate message passing with arbitrary matrices. In 2014 IEEE International Symposium on Information Theory, ISIT 2014. Institute of Electrical and Electronics Engineers Inc. 2014. p. 236-240. 6874830 https://doi.org/10.1109/ISIT.2014.6874830
Rangan, Sundeep ; Schniter, Philip ; Fletcher, Alyson. / On the convergence of approximate message passing with arbitrary matrices. 2014 IEEE International Symposium on Information Theory, ISIT 2014. Institute of Electrical and Electronics Engineers Inc., 2014. pp. 236-240
@inproceedings{cf2a17a2c18a42dfab670ba9e69cb5b3,
title = "On the convergence of approximate message passing with arbitrary matrices",
abstract = "Approximate message passing (AMP) methods and their variants have attracted considerable recent attention for the problem of estimating a random vector x observed through a linear transform A. In the case of large i.i.d. A, the methods exhibit fast convergence with precise analytic characterizations on the algorithm behavior. However, the convergence of AMP under general transforms is not fully understood. In this paper, we provide sufficient conditions for the convergence of a damped version of the generalized AMP (GAMP) algorithm in the case of Gaussian distributions. It is shown that, with sufficient damping the algorithm can be guaranteed to converge, but the amount of damping grows with peak-to-average ratio of the squared singular values of A. This condition explains the good performance of AMP methods on i.i.d. matrices, but also their difficulties with other classes of transforms. A related sufficient condition is then derived for the local stability of the damped GAMP method under more general (possibly non-Gaussian) distributions, assuming certain strict convexity conditions.",
keywords = "Belief propagation, message passing, primal-dual methods",
author = "Sundeep Rangan and Philip Schniter and Alyson Fletcher",
year = "2014",
doi = "10.1109/ISIT.2014.6874830",
language = "English (US)",
isbn = "9781479951864",
pages = "236--240",
booktitle = "2014 IEEE International Symposium on Information Theory, ISIT 2014",
publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

TY - GEN

T1 - On the convergence of approximate message passing with arbitrary matrices

AU - Rangan, Sundeep

AU - Schniter, Philip

AU - Fletcher, Alyson

PY - 2014

Y1 - 2014

N2 - Approximate message passing (AMP) methods and their variants have attracted considerable recent attention for the problem of estimating a random vector x observed through a linear transform A. In the case of large i.i.d. A, the methods exhibit fast convergence with precise analytic characterizations on the algorithm behavior. However, the convergence of AMP under general transforms is not fully understood. In this paper, we provide sufficient conditions for the convergence of a damped version of the generalized AMP (GAMP) algorithm in the case of Gaussian distributions. It is shown that, with sufficient damping the algorithm can be guaranteed to converge, but the amount of damping grows with peak-to-average ratio of the squared singular values of A. This condition explains the good performance of AMP methods on i.i.d. matrices, but also their difficulties with other classes of transforms. A related sufficient condition is then derived for the local stability of the damped GAMP method under more general (possibly non-Gaussian) distributions, assuming certain strict convexity conditions.

AB - Approximate message passing (AMP) methods and their variants have attracted considerable recent attention for the problem of estimating a random vector x observed through a linear transform A. In the case of large i.i.d. A, the methods exhibit fast convergence with precise analytic characterizations on the algorithm behavior. However, the convergence of AMP under general transforms is not fully understood. In this paper, we provide sufficient conditions for the convergence of a damped version of the generalized AMP (GAMP) algorithm in the case of Gaussian distributions. It is shown that, with sufficient damping the algorithm can be guaranteed to converge, but the amount of damping grows with peak-to-average ratio of the squared singular values of A. This condition explains the good performance of AMP methods on i.i.d. matrices, but also their difficulties with other classes of transforms. A related sufficient condition is then derived for the local stability of the damped GAMP method under more general (possibly non-Gaussian) distributions, assuming certain strict convexity conditions.

KW - Belief propagation

KW - message passing

KW - primal-dual methods

UR - http://www.scopus.com/inward/record.url?scp=84906541692&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84906541692&partnerID=8YFLogxK

U2 - 10.1109/ISIT.2014.6874830

DO - 10.1109/ISIT.2014.6874830

M3 - Conference contribution

AN - SCOPUS:84906541692

SN - 9781479951864

SP - 236

EP - 240

BT - 2014 IEEE International Symposium on Information Theory, ISIT 2014

PB - Institute of Electrical and Electronics Engineers Inc.

ER -