New outer bounds on the marginal polytope

David Sontag, Tommi Jaakkola

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We give a new class of outer bounds on the marginal polytope, and propose a cutting-plane algorithm for efficiently optimizing over these constraints. When combined with a concave upper bound on the entropy, this gives a new variational inference algorithm for probabilistic inference in discrete Markov Random Fields (MRFs). Valid constraints on the marginal polytope are derived through a series of projections onto the cut polytope. As a result, we obtain tighter upper bounds on the log-partition function. We also show empirically that the approximations of the marginals are significantly more accurate when using the tighter outer bounds. Finally, we demonstrate the advantage of the new constraints for finding the MAP assignment in protein structure prediction.

Original languageEnglish (US)
Title of host publicationAdvances in Neural Information Processing Systems 20 - Proceedings of the 2007 Conference
StatePublished - 2009
Event21st Annual Conference on Neural Information Processing Systems, NIPS 2007 - Vancouver, BC, Canada
Duration: Dec 3 2007Dec 6 2007

Other

Other21st Annual Conference on Neural Information Processing Systems, NIPS 2007
CountryCanada
CityVancouver, BC
Period12/3/0712/6/07

Fingerprint

Entropy
Proteins

ASJC Scopus subject areas

  • Information Systems

Cite this

Sontag, D., & Jaakkola, T. (2009). New outer bounds on the marginal polytope. In Advances in Neural Information Processing Systems 20 - Proceedings of the 2007 Conference

New outer bounds on the marginal polytope. / Sontag, David; Jaakkola, Tommi.

Advances in Neural Information Processing Systems 20 - Proceedings of the 2007 Conference. 2009.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Sontag, D & Jaakkola, T 2009, New outer bounds on the marginal polytope. in Advances in Neural Information Processing Systems 20 - Proceedings of the 2007 Conference. 21st Annual Conference on Neural Information Processing Systems, NIPS 2007, Vancouver, BC, Canada, 12/3/07.
Sontag D, Jaakkola T. New outer bounds on the marginal polytope. In Advances in Neural Information Processing Systems 20 - Proceedings of the 2007 Conference. 2009
Sontag, David ; Jaakkola, Tommi. / New outer bounds on the marginal polytope. Advances in Neural Information Processing Systems 20 - Proceedings of the 2007 Conference. 2009.
@inproceedings{3c8ccea5ad494c8cbba6483de37e7ecb,
title = "New outer bounds on the marginal polytope",
abstract = "We give a new class of outer bounds on the marginal polytope, and propose a cutting-plane algorithm for efficiently optimizing over these constraints. When combined with a concave upper bound on the entropy, this gives a new variational inference algorithm for probabilistic inference in discrete Markov Random Fields (MRFs). Valid constraints on the marginal polytope are derived through a series of projections onto the cut polytope. As a result, we obtain tighter upper bounds on the log-partition function. We also show empirically that the approximations of the marginals are significantly more accurate when using the tighter outer bounds. Finally, we demonstrate the advantage of the new constraints for finding the MAP assignment in protein structure prediction.",
author = "David Sontag and Tommi Jaakkola",
year = "2009",
language = "English (US)",
isbn = "160560352X",
booktitle = "Advances in Neural Information Processing Systems 20 - Proceedings of the 2007 Conference",

}

TY - GEN

T1 - New outer bounds on the marginal polytope

AU - Sontag, David

AU - Jaakkola, Tommi

PY - 2009

Y1 - 2009

N2 - We give a new class of outer bounds on the marginal polytope, and propose a cutting-plane algorithm for efficiently optimizing over these constraints. When combined with a concave upper bound on the entropy, this gives a new variational inference algorithm for probabilistic inference in discrete Markov Random Fields (MRFs). Valid constraints on the marginal polytope are derived through a series of projections onto the cut polytope. As a result, we obtain tighter upper bounds on the log-partition function. We also show empirically that the approximations of the marginals are significantly more accurate when using the tighter outer bounds. Finally, we demonstrate the advantage of the new constraints for finding the MAP assignment in protein structure prediction.

AB - We give a new class of outer bounds on the marginal polytope, and propose a cutting-plane algorithm for efficiently optimizing over these constraints. When combined with a concave upper bound on the entropy, this gives a new variational inference algorithm for probabilistic inference in discrete Markov Random Fields (MRFs). Valid constraints on the marginal polytope are derived through a series of projections onto the cut polytope. As a result, we obtain tighter upper bounds on the log-partition function. We also show empirically that the approximations of the marginals are significantly more accurate when using the tighter outer bounds. Finally, we demonstrate the advantage of the new constraints for finding the MAP assignment in protein structure prediction.

UR - http://www.scopus.com/inward/record.url?scp=84858773904&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84858773904&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:84858773904

SN - 160560352X

SN - 9781605603520

BT - Advances in Neural Information Processing Systems 20 - Proceedings of the 2007 Conference

ER -