Expectation consistent approximate inference: Generalizations and convergence

Alyson Fletcher, Mojtaba Sahraee-Ardakan, Sundeep Rangan, Philip Schniter

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Approximations of loopy belief propagation, including expectation propagation and approximate message passing, have attracted considerable attention for probabilistic inference problems. This paper proposes and analyzes a generalization of Opper and Winther's expectation consistent (EC) approximate inference method. The proposed method, called Generalized Expectation Consistency (GEC), can be applied to both maximum a posteriori (MAP) and minimum mean squared error (MMSE) estimation. Here we characterize its fixed points, convergence, and performance relative to the replica prediction of optimality.

Original languageEnglish (US)
Title of host publicationProceedings - ISIT 2016; 2016 IEEE International Symposium on Information Theory
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages190-194
Number of pages5
Volume2016-August
ISBN (Electronic)9781509018062
DOIs
StatePublished - Aug 10 2016
Event2016 IEEE International Symposium on Information Theory, ISIT 2016 - Barcelona, Spain
Duration: Jul 10 2016Jul 15 2016

Other

Other2016 IEEE International Symposium on Information Theory, ISIT 2016
CountrySpain
CityBarcelona
Period7/10/167/15/16

Fingerprint

Message passing
Error analysis
Probabilistic Inference
Belief Propagation
Maximum a Posteriori
Error Estimation
Message Passing
Replica
Mean Squared Error
Optimality
Fixed point
Propagation
Prediction
Approximation
Generalization

Keywords

  • Approximate message passing
  • Bethe free energy
  • Expectation propagation
  • S-transform in free probability

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Information Systems
  • Modeling and Simulation
  • Applied Mathematics

Cite this

Fletcher, A., Sahraee-Ardakan, M., Rangan, S., & Schniter, P. (2016). Expectation consistent approximate inference: Generalizations and convergence. In Proceedings - ISIT 2016; 2016 IEEE International Symposium on Information Theory (Vol. 2016-August, pp. 190-194). [7541287] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ISIT.2016.7541287

Expectation consistent approximate inference : Generalizations and convergence. / Fletcher, Alyson; Sahraee-Ardakan, Mojtaba; Rangan, Sundeep; Schniter, Philip.

Proceedings - ISIT 2016; 2016 IEEE International Symposium on Information Theory. Vol. 2016-August Institute of Electrical and Electronics Engineers Inc., 2016. p. 190-194 7541287.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Fletcher, A, Sahraee-Ardakan, M, Rangan, S & Schniter, P 2016, Expectation consistent approximate inference: Generalizations and convergence. in Proceedings - ISIT 2016; 2016 IEEE International Symposium on Information Theory. vol. 2016-August, 7541287, Institute of Electrical and Electronics Engineers Inc., pp. 190-194, 2016 IEEE International Symposium on Information Theory, ISIT 2016, Barcelona, Spain, 7/10/16. https://doi.org/10.1109/ISIT.2016.7541287
Fletcher A, Sahraee-Ardakan M, Rangan S, Schniter P. Expectation consistent approximate inference: Generalizations and convergence. In Proceedings - ISIT 2016; 2016 IEEE International Symposium on Information Theory. Vol. 2016-August. Institute of Electrical and Electronics Engineers Inc. 2016. p. 190-194. 7541287 https://doi.org/10.1109/ISIT.2016.7541287
Fletcher, Alyson ; Sahraee-Ardakan, Mojtaba ; Rangan, Sundeep ; Schniter, Philip. / Expectation consistent approximate inference : Generalizations and convergence. Proceedings - ISIT 2016; 2016 IEEE International Symposium on Information Theory. Vol. 2016-August Institute of Electrical and Electronics Engineers Inc., 2016. pp. 190-194
@inproceedings{d0a19fd3dfe448bea5eb9ee4ed503c32,
title = "Expectation consistent approximate inference: Generalizations and convergence",
abstract = "Approximations of loopy belief propagation, including expectation propagation and approximate message passing, have attracted considerable attention for probabilistic inference problems. This paper proposes and analyzes a generalization of Opper and Winther's expectation consistent (EC) approximate inference method. The proposed method, called Generalized Expectation Consistency (GEC), can be applied to both maximum a posteriori (MAP) and minimum mean squared error (MMSE) estimation. Here we characterize its fixed points, convergence, and performance relative to the replica prediction of optimality.",
keywords = "Approximate message passing, Bethe free energy, Expectation propagation, S-transform in free probability",
author = "Alyson Fletcher and Mojtaba Sahraee-Ardakan and Sundeep Rangan and Philip Schniter",
year = "2016",
month = "8",
day = "10",
doi = "10.1109/ISIT.2016.7541287",
language = "English (US)",
volume = "2016-August",
pages = "190--194",
booktitle = "Proceedings - ISIT 2016; 2016 IEEE International Symposium on Information Theory",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
address = "United States",

}

TY - GEN

T1 - Expectation consistent approximate inference

T2 - Generalizations and convergence

AU - Fletcher, Alyson

AU - Sahraee-Ardakan, Mojtaba

AU - Rangan, Sundeep

AU - Schniter, Philip

PY - 2016/8/10

Y1 - 2016/8/10

N2 - Approximations of loopy belief propagation, including expectation propagation and approximate message passing, have attracted considerable attention for probabilistic inference problems. This paper proposes and analyzes a generalization of Opper and Winther's expectation consistent (EC) approximate inference method. The proposed method, called Generalized Expectation Consistency (GEC), can be applied to both maximum a posteriori (MAP) and minimum mean squared error (MMSE) estimation. Here we characterize its fixed points, convergence, and performance relative to the replica prediction of optimality.

AB - Approximations of loopy belief propagation, including expectation propagation and approximate message passing, have attracted considerable attention for probabilistic inference problems. This paper proposes and analyzes a generalization of Opper and Winther's expectation consistent (EC) approximate inference method. The proposed method, called Generalized Expectation Consistency (GEC), can be applied to both maximum a posteriori (MAP) and minimum mean squared error (MMSE) estimation. Here we characterize its fixed points, convergence, and performance relative to the replica prediction of optimality.

KW - Approximate message passing

KW - Bethe free energy

KW - Expectation propagation

KW - S-transform in free probability

UR - http://www.scopus.com/inward/record.url?scp=84985987005&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84985987005&partnerID=8YFLogxK

U2 - 10.1109/ISIT.2016.7541287

DO - 10.1109/ISIT.2016.7541287

M3 - Conference contribution

AN - SCOPUS:84985987005

VL - 2016-August

SP - 190

EP - 194

BT - Proceedings - ISIT 2016; 2016 IEEE International Symposium on Information Theory

PB - Institute of Electrical and Electronics Engineers Inc.

ER -