Understanding the Bethe approximation

When and how can it go wrong?

Adrian Weller, Kui Tang, David Sontag, Tony Jebara

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Belief propagation is a remarkably effective tool for inference, even when applied to networks with cycles. It may be viewed as a way to seek the minimum of the Bethe free energy, though with no convergence guarantee in general. A variational perspective shows that, compared to exact inference, this minimization employs two forms of approximation: (i) the true entropy is approximated by the Bethe entropy, and (ii) the minimization is performed over a relaxation of the marginal polytope termed the local polytope. Here we explore when and how the Bethe approximation can fail for binary pairwise models by examining each aspect of the approximation, deriving results both analytically and with new experimental methods.

Original languageEnglish (US)
Title of host publicationUncertainty in Artificial Intelligence - Proceedings of the 30th Conference, UAI 2014
PublisherAUAI Press
Pages868-877
Number of pages10
ISBN (Print)9780974903910
StatePublished - 2014
Event30th Conference on Uncertainty in Artificial Intelligence, UAI 2014 - Quebec City, Canada
Duration: Jul 23 2014Jul 27 2014

Other

Other30th Conference on Uncertainty in Artificial Intelligence, UAI 2014
CountryCanada
CityQuebec City
Period7/23/147/27/14

Fingerprint

Entropy
Free energy

ASJC Scopus subject areas

  • Artificial Intelligence

Cite this

Weller, A., Tang, K., Sontag, D., & Jebara, T. (2014). Understanding the Bethe approximation: When and how can it go wrong? In Uncertainty in Artificial Intelligence - Proceedings of the 30th Conference, UAI 2014 (pp. 868-877). AUAI Press.

Understanding the Bethe approximation : When and how can it go wrong? / Weller, Adrian; Tang, Kui; Sontag, David; Jebara, Tony.

Uncertainty in Artificial Intelligence - Proceedings of the 30th Conference, UAI 2014. AUAI Press, 2014. p. 868-877.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Weller, A, Tang, K, Sontag, D & Jebara, T 2014, Understanding the Bethe approximation: When and how can it go wrong? in Uncertainty in Artificial Intelligence - Proceedings of the 30th Conference, UAI 2014. AUAI Press, pp. 868-877, 30th Conference on Uncertainty in Artificial Intelligence, UAI 2014, Quebec City, Canada, 7/23/14.
Weller A, Tang K, Sontag D, Jebara T. Understanding the Bethe approximation: When and how can it go wrong? In Uncertainty in Artificial Intelligence - Proceedings of the 30th Conference, UAI 2014. AUAI Press. 2014. p. 868-877
Weller, Adrian ; Tang, Kui ; Sontag, David ; Jebara, Tony. / Understanding the Bethe approximation : When and how can it go wrong?. Uncertainty in Artificial Intelligence - Proceedings of the 30th Conference, UAI 2014. AUAI Press, 2014. pp. 868-877
@inproceedings{790806af221b48dbb80f0b77de373398,
title = "Understanding the Bethe approximation: When and how can it go wrong?",
abstract = "Belief propagation is a remarkably effective tool for inference, even when applied to networks with cycles. It may be viewed as a way to seek the minimum of the Bethe free energy, though with no convergence guarantee in general. A variational perspective shows that, compared to exact inference, this minimization employs two forms of approximation: (i) the true entropy is approximated by the Bethe entropy, and (ii) the minimization is performed over a relaxation of the marginal polytope termed the local polytope. Here we explore when and how the Bethe approximation can fail for binary pairwise models by examining each aspect of the approximation, deriving results both analytically and with new experimental methods.",
author = "Adrian Weller and Kui Tang and David Sontag and Tony Jebara",
year = "2014",
language = "English (US)",
isbn = "9780974903910",
pages = "868--877",
booktitle = "Uncertainty in Artificial Intelligence - Proceedings of the 30th Conference, UAI 2014",
publisher = "AUAI Press",

}

TY - GEN

T1 - Understanding the Bethe approximation

T2 - When and how can it go wrong?

AU - Weller, Adrian

AU - Tang, Kui

AU - Sontag, David

AU - Jebara, Tony

PY - 2014

Y1 - 2014

N2 - Belief propagation is a remarkably effective tool for inference, even when applied to networks with cycles. It may be viewed as a way to seek the minimum of the Bethe free energy, though with no convergence guarantee in general. A variational perspective shows that, compared to exact inference, this minimization employs two forms of approximation: (i) the true entropy is approximated by the Bethe entropy, and (ii) the minimization is performed over a relaxation of the marginal polytope termed the local polytope. Here we explore when and how the Bethe approximation can fail for binary pairwise models by examining each aspect of the approximation, deriving results both analytically and with new experimental methods.

AB - Belief propagation is a remarkably effective tool for inference, even when applied to networks with cycles. It may be viewed as a way to seek the minimum of the Bethe free energy, though with no convergence guarantee in general. A variational perspective shows that, compared to exact inference, this minimization employs two forms of approximation: (i) the true entropy is approximated by the Bethe entropy, and (ii) the minimization is performed over a relaxation of the marginal polytope termed the local polytope. Here we explore when and how the Bethe approximation can fail for binary pairwise models by examining each aspect of the approximation, deriving results both analytically and with new experimental methods.

UR - http://www.scopus.com/inward/record.url?scp=84923278848&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84923278848&partnerID=8YFLogxK

M3 - Conference contribution

SN - 9780974903910

SP - 868

EP - 877

BT - Uncertainty in Artificial Intelligence - Proceedings of the 30th Conference, UAI 2014

PB - AUAI Press

ER -