Fixed points of generalized approximate message passing with arbitrary matrices

Sundeep Rangan, Philip Schniter, Erwin Riegler, Alyson K. Fletcher, Volkan Cevher

Research output: Contribution to journalArticle

Abstract

The estimation of a random vector with independent components passed through a linear transform followed by a componentwise (possibly nonlinear) output map arises in a range of applications. Approximate message passing (AMP) methods, based on Gaussian approximations of loopy belief propagation, have recently attracted considerable attention for such problems. For large random transforms, these methods exhibit fast convergence and admit precise analytic characterizations with testable conditions for optimality, even for certain non-convex problem instances. However, the behavior of AMP under general transforms is not fully understood. In this paper, we consider the generalized AMP (GAMP) algorithm and relate the method to more common optimization techniques. This analysis enables a precise characterization of the GAMP algorithm fixed points that applies to arbitrary transforms. In particular, we show that the fixed points of the so-called max-sum GAMP algorithm for MAP estimation are critical points of a constrained maximization of the posterior density. The fixed points of the sum-product GAMP algorithm for estimation of the posterior marginals can be interpreted as critical points of a certain free energy.

Original languageEnglish (US)
Article number7600404
Pages (from-to)7464-7474
Number of pages11
JournalIEEE Transactions on Information Theory
Volume62
Issue number12
DOIs
StatePublished - Dec 1 2016

Fingerprint

Message passing
Free energy
energy

Keywords

  • ADMM
  • belief propagation
  • compressed sensing
  • Message passing
  • variational optimization

ASJC Scopus subject areas

  • Information Systems
  • Computer Science Applications
  • Library and Information Sciences

Cite this

Fixed points of generalized approximate message passing with arbitrary matrices. / Rangan, Sundeep; Schniter, Philip; Riegler, Erwin; Fletcher, Alyson K.; Cevher, Volkan.

In: IEEE Transactions on Information Theory, Vol. 62, No. 12, 7600404, 01.12.2016, p. 7464-7474.

Research output: Contribution to journalArticle

Rangan, Sundeep ; Schniter, Philip ; Riegler, Erwin ; Fletcher, Alyson K. ; Cevher, Volkan. / Fixed points of generalized approximate message passing with arbitrary matrices. In: IEEE Transactions on Information Theory. 2016 ; Vol. 62, No. 12. pp. 7464-7474.
@article{15d5d2b0f7954279bf966ea9cab583f2,
title = "Fixed points of generalized approximate message passing with arbitrary matrices",
abstract = "The estimation of a random vector with independent components passed through a linear transform followed by a componentwise (possibly nonlinear) output map arises in a range of applications. Approximate message passing (AMP) methods, based on Gaussian approximations of loopy belief propagation, have recently attracted considerable attention for such problems. For large random transforms, these methods exhibit fast convergence and admit precise analytic characterizations with testable conditions for optimality, even for certain non-convex problem instances. However, the behavior of AMP under general transforms is not fully understood. In this paper, we consider the generalized AMP (GAMP) algorithm and relate the method to more common optimization techniques. This analysis enables a precise characterization of the GAMP algorithm fixed points that applies to arbitrary transforms. In particular, we show that the fixed points of the so-called max-sum GAMP algorithm for MAP estimation are critical points of a constrained maximization of the posterior density. The fixed points of the sum-product GAMP algorithm for estimation of the posterior marginals can be interpreted as critical points of a certain free energy.",
keywords = "ADMM, belief propagation, compressed sensing, Message passing, variational optimization",
author = "Sundeep Rangan and Philip Schniter and Erwin Riegler and Fletcher, {Alyson K.} and Volkan Cevher",
year = "2016",
month = "12",
day = "1",
doi = "10.1109/TIT.2016.2619365",
language = "English (US)",
volume = "62",
pages = "7464--7474",
journal = "IEEE Transactions on Information Theory",
issn = "0018-9448",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
number = "12",

}

TY - JOUR

T1 - Fixed points of generalized approximate message passing with arbitrary matrices

AU - Rangan, Sundeep

AU - Schniter, Philip

AU - Riegler, Erwin

AU - Fletcher, Alyson K.

AU - Cevher, Volkan

PY - 2016/12/1

Y1 - 2016/12/1

N2 - The estimation of a random vector with independent components passed through a linear transform followed by a componentwise (possibly nonlinear) output map arises in a range of applications. Approximate message passing (AMP) methods, based on Gaussian approximations of loopy belief propagation, have recently attracted considerable attention for such problems. For large random transforms, these methods exhibit fast convergence and admit precise analytic characterizations with testable conditions for optimality, even for certain non-convex problem instances. However, the behavior of AMP under general transforms is not fully understood. In this paper, we consider the generalized AMP (GAMP) algorithm and relate the method to more common optimization techniques. This analysis enables a precise characterization of the GAMP algorithm fixed points that applies to arbitrary transforms. In particular, we show that the fixed points of the so-called max-sum GAMP algorithm for MAP estimation are critical points of a constrained maximization of the posterior density. The fixed points of the sum-product GAMP algorithm for estimation of the posterior marginals can be interpreted as critical points of a certain free energy.

AB - The estimation of a random vector with independent components passed through a linear transform followed by a componentwise (possibly nonlinear) output map arises in a range of applications. Approximate message passing (AMP) methods, based on Gaussian approximations of loopy belief propagation, have recently attracted considerable attention for such problems. For large random transforms, these methods exhibit fast convergence and admit precise analytic characterizations with testable conditions for optimality, even for certain non-convex problem instances. However, the behavior of AMP under general transforms is not fully understood. In this paper, we consider the generalized AMP (GAMP) algorithm and relate the method to more common optimization techniques. This analysis enables a precise characterization of the GAMP algorithm fixed points that applies to arbitrary transforms. In particular, we show that the fixed points of the so-called max-sum GAMP algorithm for MAP estimation are critical points of a constrained maximization of the posterior density. The fixed points of the sum-product GAMP algorithm for estimation of the posterior marginals can be interpreted as critical points of a certain free energy.

KW - ADMM

KW - belief propagation

KW - compressed sensing

KW - Message passing

KW - variational optimization

UR - http://www.scopus.com/inward/record.url?scp=85000796404&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85000796404&partnerID=8YFLogxK

U2 - 10.1109/TIT.2016.2619365

DO - 10.1109/TIT.2016.2619365

M3 - Article

VL - 62

SP - 7464

EP - 7474

JO - IEEE Transactions on Information Theory

JF - IEEE Transactions on Information Theory

SN - 0018-9448

IS - 12

M1 - 7600404

ER -