Hybrid Approximate Message Passing

Sundeep Rangan, Alyson K. Fletcher, Vivek K. Goyal, Evan Byrne, Philip Schniter

Research output: Contribution to journalArticle

Abstract

Gaussian and quadratic approximations of message passing algorithms on graphs have attracted considerable recent attention due to their computational simplicity, analytic tractability, and wide applicability in optimization and statistical inference problems. This paper presents a systematic framework for incorporating such approximate message passing (AMP) methods in general graphical models. The key concept is a partition of dependencies of a general graphical model into strong and weak edges, with the weak edges representing small, linearizable couplings of variables. AMP approximations based on the central limit theorem can be readily applied to aggregates of many weak edges and integrated with standard message passing updates on the strong edges. The resulting algorithm, which we call hybrid generalized approximate message passing (HyGAMP), can yield significantly simpler implementations of sum-product and max-sum loopy belief propagation. By varying the partition of strong and weak edges, a performance-complexity tradeoff can be achieved. Group sparsity and multinomial logistic regression problems are studied as examples of the proposed methodology.

Original languageEnglish (US)
Article number7944630
Pages (from-to)4577-4592
Number of pages16
JournalIEEE Transactions on Signal Processing
Volume65
Issue number17
DOIs
StatePublished - Sep 1 2017

Fingerprint

Message passing
Logistics

Keywords

  • Approximate message passing
  • belief propagation
  • group sparsity
  • max-sum algorithm
  • multinomial logistic regression
  • sum-product algorithm

ASJC Scopus subject areas

  • Signal Processing
  • Electrical and Electronic Engineering

Cite this

Rangan, S., Fletcher, A. K., Goyal, V. K., Byrne, E., & Schniter, P. (2017). Hybrid Approximate Message Passing. IEEE Transactions on Signal Processing, 65(17), 4577-4592. [7944630]. https://doi.org/10.1109/TSP.2017.2713759

Hybrid Approximate Message Passing. / Rangan, Sundeep; Fletcher, Alyson K.; Goyal, Vivek K.; Byrne, Evan; Schniter, Philip.

In: IEEE Transactions on Signal Processing, Vol. 65, No. 17, 7944630, 01.09.2017, p. 4577-4592.

Research output: Contribution to journalArticle

Rangan, S, Fletcher, AK, Goyal, VK, Byrne, E & Schniter, P 2017, 'Hybrid Approximate Message Passing', IEEE Transactions on Signal Processing, vol. 65, no. 17, 7944630, pp. 4577-4592. https://doi.org/10.1109/TSP.2017.2713759
Rangan S, Fletcher AK, Goyal VK, Byrne E, Schniter P. Hybrid Approximate Message Passing. IEEE Transactions on Signal Processing. 2017 Sep 1;65(17):4577-4592. 7944630. https://doi.org/10.1109/TSP.2017.2713759
Rangan, Sundeep ; Fletcher, Alyson K. ; Goyal, Vivek K. ; Byrne, Evan ; Schniter, Philip. / Hybrid Approximate Message Passing. In: IEEE Transactions on Signal Processing. 2017 ; Vol. 65, No. 17. pp. 4577-4592.
@article{65062e0da3d4443c99d7c1a895b62e43,
title = "Hybrid Approximate Message Passing",
abstract = "Gaussian and quadratic approximations of message passing algorithms on graphs have attracted considerable recent attention due to their computational simplicity, analytic tractability, and wide applicability in optimization and statistical inference problems. This paper presents a systematic framework for incorporating such approximate message passing (AMP) methods in general graphical models. The key concept is a partition of dependencies of a general graphical model into strong and weak edges, with the weak edges representing small, linearizable couplings of variables. AMP approximations based on the central limit theorem can be readily applied to aggregates of many weak edges and integrated with standard message passing updates on the strong edges. The resulting algorithm, which we call hybrid generalized approximate message passing (HyGAMP), can yield significantly simpler implementations of sum-product and max-sum loopy belief propagation. By varying the partition of strong and weak edges, a performance-complexity tradeoff can be achieved. Group sparsity and multinomial logistic regression problems are studied as examples of the proposed methodology.",
keywords = "Approximate message passing, belief propagation, group sparsity, max-sum algorithm, multinomial logistic regression, sum-product algorithm",
author = "Sundeep Rangan and Fletcher, {Alyson K.} and Goyal, {Vivek K.} and Evan Byrne and Philip Schniter",
year = "2017",
month = "9",
day = "1",
doi = "10.1109/TSP.2017.2713759",
language = "English (US)",
volume = "65",
pages = "4577--4592",
journal = "IEEE Transactions on Signal Processing",
issn = "1053-587X",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
number = "17",

}

TY - JOUR

T1 - Hybrid Approximate Message Passing

AU - Rangan, Sundeep

AU - Fletcher, Alyson K.

AU - Goyal, Vivek K.

AU - Byrne, Evan

AU - Schniter, Philip

PY - 2017/9/1

Y1 - 2017/9/1

N2 - Gaussian and quadratic approximations of message passing algorithms on graphs have attracted considerable recent attention due to their computational simplicity, analytic tractability, and wide applicability in optimization and statistical inference problems. This paper presents a systematic framework for incorporating such approximate message passing (AMP) methods in general graphical models. The key concept is a partition of dependencies of a general graphical model into strong and weak edges, with the weak edges representing small, linearizable couplings of variables. AMP approximations based on the central limit theorem can be readily applied to aggregates of many weak edges and integrated with standard message passing updates on the strong edges. The resulting algorithm, which we call hybrid generalized approximate message passing (HyGAMP), can yield significantly simpler implementations of sum-product and max-sum loopy belief propagation. By varying the partition of strong and weak edges, a performance-complexity tradeoff can be achieved. Group sparsity and multinomial logistic regression problems are studied as examples of the proposed methodology.

AB - Gaussian and quadratic approximations of message passing algorithms on graphs have attracted considerable recent attention due to their computational simplicity, analytic tractability, and wide applicability in optimization and statistical inference problems. This paper presents a systematic framework for incorporating such approximate message passing (AMP) methods in general graphical models. The key concept is a partition of dependencies of a general graphical model into strong and weak edges, with the weak edges representing small, linearizable couplings of variables. AMP approximations based on the central limit theorem can be readily applied to aggregates of many weak edges and integrated with standard message passing updates on the strong edges. The resulting algorithm, which we call hybrid generalized approximate message passing (HyGAMP), can yield significantly simpler implementations of sum-product and max-sum loopy belief propagation. By varying the partition of strong and weak edges, a performance-complexity tradeoff can be achieved. Group sparsity and multinomial logistic regression problems are studied as examples of the proposed methodology.

KW - Approximate message passing

KW - belief propagation

KW - group sparsity

KW - max-sum algorithm

KW - multinomial logistic regression

KW - sum-product algorithm

UR - http://www.scopus.com/inward/record.url?scp=85028385903&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85028385903&partnerID=8YFLogxK

U2 - 10.1109/TSP.2017.2713759

DO - 10.1109/TSP.2017.2713759

M3 - Article

AN - SCOPUS:85028385903

VL - 65

SP - 4577

EP - 4592

JO - IEEE Transactions on Signal Processing

JF - IEEE Transactions on Signal Processing

SN - 1053-587X

IS - 17

M1 - 7944630

ER -