Hybrid generalized approximate message passing with applications to structured sparsity

Sundeep Rangan, Alyson K. Fletcher, Vivek K. Goyal, Philip Schniter

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Gaussian and quadratic approximations of message passing algorithms on graphs have attracted considerable attention due to their computational simplicity, analytic tractability, and wide applicability in optimization and statistical inference problems. This paper summarizes a systematic framework for incorporating such approximate message passing (AMP) methods in general graphical models. The key concept is a partition of dependencies of a general graphical model into strong and weak edges, with each weak edge representing a small, linearizable coupling of variables. AMP approximations based on the central limit theorem can be applied to the weak edges and integrated with standard message passing updates on the strong edges. The resulting algorithm, which we call hybrid generalized approximate message passing (Hybrid-GAMP), can yield significantly simpler implementations of sum-product and max-sum loopy belief propagation. By varying the partition between strong and weak edges, a performance-complexity trade-off can be achieved. Structured sparsity problems are studied as an example of this general methodology where there is a natural partition of edges.

Original languageEnglish (US)
Title of host publication2012 IEEE International Symposium on Information Theory Proceedings, ISIT 2012
Pages1236-1240
Number of pages5
DOIs
StatePublished - 2012
Event2012 IEEE International Symposium on Information Theory, ISIT 2012 - Cambridge, MA, United States
Duration: Jul 1 2012Jul 6 2012

Other

Other2012 IEEE International Symposium on Information Theory, ISIT 2012
CountryUnited States
CityCambridge, MA
Period7/1/127/6/12

Fingerprint

Message passing
Message Passing
Sparsity
Partition
Graphical Models
Message-passing Algorithms
Quadratic Approximation
Gaussian Approximation
Belief Propagation
Tractability
Statistical Inference
Central limit theorem
Simplicity
Trade-offs
Update
Optimization
Methodology
Approximation
Graph in graph theory

ASJC Scopus subject areas

  • Applied Mathematics
  • Modeling and Simulation
  • Theoretical Computer Science
  • Information Systems

Cite this

Rangan, S., Fletcher, A. K., Goyal, V. K., & Schniter, P. (2012). Hybrid generalized approximate message passing with applications to structured sparsity. In 2012 IEEE International Symposium on Information Theory Proceedings, ISIT 2012 (pp. 1236-1240). [6283054] https://doi.org/10.1109/ISIT.2012.6283054

Hybrid generalized approximate message passing with applications to structured sparsity. / Rangan, Sundeep; Fletcher, Alyson K.; Goyal, Vivek K.; Schniter, Philip.

2012 IEEE International Symposium on Information Theory Proceedings, ISIT 2012. 2012. p. 1236-1240 6283054.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Rangan, S, Fletcher, AK, Goyal, VK & Schniter, P 2012, Hybrid generalized approximate message passing with applications to structured sparsity. in 2012 IEEE International Symposium on Information Theory Proceedings, ISIT 2012., 6283054, pp. 1236-1240, 2012 IEEE International Symposium on Information Theory, ISIT 2012, Cambridge, MA, United States, 7/1/12. https://doi.org/10.1109/ISIT.2012.6283054
Rangan S, Fletcher AK, Goyal VK, Schniter P. Hybrid generalized approximate message passing with applications to structured sparsity. In 2012 IEEE International Symposium on Information Theory Proceedings, ISIT 2012. 2012. p. 1236-1240. 6283054 https://doi.org/10.1109/ISIT.2012.6283054
Rangan, Sundeep ; Fletcher, Alyson K. ; Goyal, Vivek K. ; Schniter, Philip. / Hybrid generalized approximate message passing with applications to structured sparsity. 2012 IEEE International Symposium on Information Theory Proceedings, ISIT 2012. 2012. pp. 1236-1240
@inproceedings{01a5f7816a114755863223d067cbab38,
title = "Hybrid generalized approximate message passing with applications to structured sparsity",
abstract = "Gaussian and quadratic approximations of message passing algorithms on graphs have attracted considerable attention due to their computational simplicity, analytic tractability, and wide applicability in optimization and statistical inference problems. This paper summarizes a systematic framework for incorporating such approximate message passing (AMP) methods in general graphical models. The key concept is a partition of dependencies of a general graphical model into strong and weak edges, with each weak edge representing a small, linearizable coupling of variables. AMP approximations based on the central limit theorem can be applied to the weak edges and integrated with standard message passing updates on the strong edges. The resulting algorithm, which we call hybrid generalized approximate message passing (Hybrid-GAMP), can yield significantly simpler implementations of sum-product and max-sum loopy belief propagation. By varying the partition between strong and weak edges, a performance-complexity trade-off can be achieved. Structured sparsity problems are studied as an example of this general methodology where there is a natural partition of edges.",
author = "Sundeep Rangan and Fletcher, {Alyson K.} and Goyal, {Vivek K.} and Philip Schniter",
year = "2012",
doi = "10.1109/ISIT.2012.6283054",
language = "English (US)",
isbn = "9781467325790",
pages = "1236--1240",
booktitle = "2012 IEEE International Symposium on Information Theory Proceedings, ISIT 2012",

}

TY - GEN

T1 - Hybrid generalized approximate message passing with applications to structured sparsity

AU - Rangan, Sundeep

AU - Fletcher, Alyson K.

AU - Goyal, Vivek K.

AU - Schniter, Philip

PY - 2012

Y1 - 2012

N2 - Gaussian and quadratic approximations of message passing algorithms on graphs have attracted considerable attention due to their computational simplicity, analytic tractability, and wide applicability in optimization and statistical inference problems. This paper summarizes a systematic framework for incorporating such approximate message passing (AMP) methods in general graphical models. The key concept is a partition of dependencies of a general graphical model into strong and weak edges, with each weak edge representing a small, linearizable coupling of variables. AMP approximations based on the central limit theorem can be applied to the weak edges and integrated with standard message passing updates on the strong edges. The resulting algorithm, which we call hybrid generalized approximate message passing (Hybrid-GAMP), can yield significantly simpler implementations of sum-product and max-sum loopy belief propagation. By varying the partition between strong and weak edges, a performance-complexity trade-off can be achieved. Structured sparsity problems are studied as an example of this general methodology where there is a natural partition of edges.

AB - Gaussian and quadratic approximations of message passing algorithms on graphs have attracted considerable attention due to their computational simplicity, analytic tractability, and wide applicability in optimization and statistical inference problems. This paper summarizes a systematic framework for incorporating such approximate message passing (AMP) methods in general graphical models. The key concept is a partition of dependencies of a general graphical model into strong and weak edges, with each weak edge representing a small, linearizable coupling of variables. AMP approximations based on the central limit theorem can be applied to the weak edges and integrated with standard message passing updates on the strong edges. The resulting algorithm, which we call hybrid generalized approximate message passing (Hybrid-GAMP), can yield significantly simpler implementations of sum-product and max-sum loopy belief propagation. By varying the partition between strong and weak edges, a performance-complexity trade-off can be achieved. Structured sparsity problems are studied as an example of this general methodology where there is a natural partition of edges.

UR - http://www.scopus.com/inward/record.url?scp=84867515003&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84867515003&partnerID=8YFLogxK

U2 - 10.1109/ISIT.2012.6283054

DO - 10.1109/ISIT.2012.6283054

M3 - Conference contribution

AN - SCOPUS:84867515003

SN - 9781467325790

SP - 1236

EP - 1240

BT - 2012 IEEE International Symposium on Information Theory Proceedings, ISIT 2012

ER -