On dual decomposition and linear programming relaxations for natural language processing

Alexander M. Rush, David Sontag, Michael Collins, Tommi Jaakkola

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

This paper introduces dual decomposition as a framework for deriving inference algorithms for NLP problems. The approach relies on standard dynamic-programming algorithms as oracle solvers for sub-problems, together with a simple method for forcing agreement between the different oracles. The approach provably solves a linear programming (LP) relaxation of the global inference problem. It leads to algorithms that are simple, in that they use existing decoding algorithms; efficient, in that they avoid exact algorithms for the full model; and often exact, in that empirically they often recover the correct solution in spite of using an LP relaxation. We give experimental results on two problems: 1) the combination of two lexicalized parsing models; and 2) the combination of a lexicalized parsing model and a trigram part-of-speech tagger.

Original languageEnglish (US)
Title of host publicationEMNLP 2010 - Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference
Pages1-11
Number of pages11
StatePublished - 2010
EventConference on Empirical Methods in Natural Language Processing, EMNLP 2010 - Cambridge, MA, United States
Duration: Oct 9 2010Oct 11 2010

Other

OtherConference on Empirical Methods in Natural Language Processing, EMNLP 2010
CountryUnited States
CityCambridge, MA
Period10/9/1010/11/10

Fingerprint

Linear programming
Decomposition
Processing
Dynamic programming
Decoding

ASJC Scopus subject areas

  • Computational Theory and Mathematics
  • Computer Science Applications
  • Information Systems

Cite this

Rush, A. M., Sontag, D., Collins, M., & Jaakkola, T. (2010). On dual decomposition and linear programming relaxations for natural language processing. In EMNLP 2010 - Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference (pp. 1-11)

On dual decomposition and linear programming relaxations for natural language processing. / Rush, Alexander M.; Sontag, David; Collins, Michael; Jaakkola, Tommi.

EMNLP 2010 - Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference. 2010. p. 1-11.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Rush, AM, Sontag, D, Collins, M & Jaakkola, T 2010, On dual decomposition and linear programming relaxations for natural language processing. in EMNLP 2010 - Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference. pp. 1-11, Conference on Empirical Methods in Natural Language Processing, EMNLP 2010, Cambridge, MA, United States, 10/9/10.
Rush AM, Sontag D, Collins M, Jaakkola T. On dual decomposition and linear programming relaxations for natural language processing. In EMNLP 2010 - Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference. 2010. p. 1-11
Rush, Alexander M. ; Sontag, David ; Collins, Michael ; Jaakkola, Tommi. / On dual decomposition and linear programming relaxations for natural language processing. EMNLP 2010 - Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference. 2010. pp. 1-11
@inproceedings{0b2acaccd39d4714998790be54bec034,
title = "On dual decomposition and linear programming relaxations for natural language processing",
abstract = "This paper introduces dual decomposition as a framework for deriving inference algorithms for NLP problems. The approach relies on standard dynamic-programming algorithms as oracle solvers for sub-problems, together with a simple method for forcing agreement between the different oracles. The approach provably solves a linear programming (LP) relaxation of the global inference problem. It leads to algorithms that are simple, in that they use existing decoding algorithms; efficient, in that they avoid exact algorithms for the full model; and often exact, in that empirically they often recover the correct solution in spite of using an LP relaxation. We give experimental results on two problems: 1) the combination of two lexicalized parsing models; and 2) the combination of a lexicalized parsing model and a trigram part-of-speech tagger.",
author = "Rush, {Alexander M.} and David Sontag and Michael Collins and Tommi Jaakkola",
year = "2010",
language = "English (US)",
isbn = "1932432868",
pages = "1--11",
booktitle = "EMNLP 2010 - Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference",

}

TY - GEN

T1 - On dual decomposition and linear programming relaxations for natural language processing

AU - Rush, Alexander M.

AU - Sontag, David

AU - Collins, Michael

AU - Jaakkola, Tommi

PY - 2010

Y1 - 2010

N2 - This paper introduces dual decomposition as a framework for deriving inference algorithms for NLP problems. The approach relies on standard dynamic-programming algorithms as oracle solvers for sub-problems, together with a simple method for forcing agreement between the different oracles. The approach provably solves a linear programming (LP) relaxation of the global inference problem. It leads to algorithms that are simple, in that they use existing decoding algorithms; efficient, in that they avoid exact algorithms for the full model; and often exact, in that empirically they often recover the correct solution in spite of using an LP relaxation. We give experimental results on two problems: 1) the combination of two lexicalized parsing models; and 2) the combination of a lexicalized parsing model and a trigram part-of-speech tagger.

AB - This paper introduces dual decomposition as a framework for deriving inference algorithms for NLP problems. The approach relies on standard dynamic-programming algorithms as oracle solvers for sub-problems, together with a simple method for forcing agreement between the different oracles. The approach provably solves a linear programming (LP) relaxation of the global inference problem. It leads to algorithms that are simple, in that they use existing decoding algorithms; efficient, in that they avoid exact algorithms for the full model; and often exact, in that empirically they often recover the correct solution in spite of using an LP relaxation. We give experimental results on two problems: 1) the combination of two lexicalized parsing models; and 2) the combination of a lexicalized parsing model and a trigram part-of-speech tagger.

UR - http://www.scopus.com/inward/record.url?scp=80053227279&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=80053227279&partnerID=8YFLogxK

M3 - Conference contribution

SN - 1932432868

SN - 9781932432862

SP - 1

EP - 11

BT - EMNLP 2010 - Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference

ER -