Classification-based strategies for combining multiple 5-W question answering systems

Sibel Yaman, Dilek Hakkani-Tur, Gokhan Tur, Ralph Grishman, Mary Harper, Kathleen R. McKeown, Adam Meyers, Kartavya Sharma

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We describe and analyze inference strategies for combining outputs from multiple question answering systems each of which was developed independently. Specifically, we address the DARPA-funded GALE information distillation Year 3 task of finding answers to the 5-Wh questions (who, what, when, where, and why) for each given sentence. The approach we take revolves around determining the best system using discriminative learning. In particular, we train support vector machines with a set of novel features that encode systems' capabilities of returning as many correct answers as possible. We analyze two combination strategies: one combines multiple systems at the granularity of sentences, and the other at the granularity of individual fields. Our experimental results indicate that the proposed features and combination strategies were able to improve the overall performance by 22% to 36% relative to a random selection, 16% to 35% relative to a majority voting scheme, and 15% to 23% relative to the best individual system.

Original languageEnglish (US)
Title of host publicationProceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH
Pages2703-2706
Number of pages4
StatePublished - 2009
Event10th Annual Conference of the International Speech Communication Association, INTERSPEECH 2009 - Brighton, United Kingdom
Duration: Sep 6 2009Sep 10 2009

Other

Other10th Annual Conference of the International Speech Communication Association, INTERSPEECH 2009
CountryUnited Kingdom
CityBrighton
Period9/6/099/10/09

Fingerprint

Distillation
Politics
Support vector machines
Learning
Support Vector Machine

Keywords

  • Question answering
  • Systems for spoken language understanding

ASJC Scopus subject areas

  • Human-Computer Interaction
  • Signal Processing
  • Software
  • Sensory Systems

Cite this

Yaman, S., Hakkani-Tur, D., Tur, G., Grishman, R., Harper, M., McKeown, K. R., ... Sharma, K. (2009). Classification-based strategies for combining multiple 5-W question answering systems. In Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH (pp. 2703-2706)

Classification-based strategies for combining multiple 5-W question answering systems. / Yaman, Sibel; Hakkani-Tur, Dilek; Tur, Gokhan; Grishman, Ralph; Harper, Mary; McKeown, Kathleen R.; Meyers, Adam; Sharma, Kartavya.

Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH. 2009. p. 2703-2706.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Yaman, S, Hakkani-Tur, D, Tur, G, Grishman, R, Harper, M, McKeown, KR, Meyers, A & Sharma, K 2009, Classification-based strategies for combining multiple 5-W question answering systems. in Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH. pp. 2703-2706, 10th Annual Conference of the International Speech Communication Association, INTERSPEECH 2009, Brighton, United Kingdom, 9/6/09.
Yaman S, Hakkani-Tur D, Tur G, Grishman R, Harper M, McKeown KR et al. Classification-based strategies for combining multiple 5-W question answering systems. In Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH. 2009. p. 2703-2706
Yaman, Sibel ; Hakkani-Tur, Dilek ; Tur, Gokhan ; Grishman, Ralph ; Harper, Mary ; McKeown, Kathleen R. ; Meyers, Adam ; Sharma, Kartavya. / Classification-based strategies for combining multiple 5-W question answering systems. Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH. 2009. pp. 2703-2706
@inproceedings{0958507267cc4f1d91673dae5e5c8238,
title = "Classification-based strategies for combining multiple 5-W question answering systems",
abstract = "We describe and analyze inference strategies for combining outputs from multiple question answering systems each of which was developed independently. Specifically, we address the DARPA-funded GALE information distillation Year 3 task of finding answers to the 5-Wh questions (who, what, when, where, and why) for each given sentence. The approach we take revolves around determining the best system using discriminative learning. In particular, we train support vector machines with a set of novel features that encode systems' capabilities of returning as many correct answers as possible. We analyze two combination strategies: one combines multiple systems at the granularity of sentences, and the other at the granularity of individual fields. Our experimental results indicate that the proposed features and combination strategies were able to improve the overall performance by 22{\%} to 36{\%} relative to a random selection, 16{\%} to 35{\%} relative to a majority voting scheme, and 15{\%} to 23{\%} relative to the best individual system.",
keywords = "Question answering, Systems for spoken language understanding",
author = "Sibel Yaman and Dilek Hakkani-Tur and Gokhan Tur and Ralph Grishman and Mary Harper and McKeown, {Kathleen R.} and Adam Meyers and Kartavya Sharma",
year = "2009",
language = "English (US)",
pages = "2703--2706",
booktitle = "Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH",

}

TY - GEN

T1 - Classification-based strategies for combining multiple 5-W question answering systems

AU - Yaman, Sibel

AU - Hakkani-Tur, Dilek

AU - Tur, Gokhan

AU - Grishman, Ralph

AU - Harper, Mary

AU - McKeown, Kathleen R.

AU - Meyers, Adam

AU - Sharma, Kartavya

PY - 2009

Y1 - 2009

N2 - We describe and analyze inference strategies for combining outputs from multiple question answering systems each of which was developed independently. Specifically, we address the DARPA-funded GALE information distillation Year 3 task of finding answers to the 5-Wh questions (who, what, when, where, and why) for each given sentence. The approach we take revolves around determining the best system using discriminative learning. In particular, we train support vector machines with a set of novel features that encode systems' capabilities of returning as many correct answers as possible. We analyze two combination strategies: one combines multiple systems at the granularity of sentences, and the other at the granularity of individual fields. Our experimental results indicate that the proposed features and combination strategies were able to improve the overall performance by 22% to 36% relative to a random selection, 16% to 35% relative to a majority voting scheme, and 15% to 23% relative to the best individual system.

AB - We describe and analyze inference strategies for combining outputs from multiple question answering systems each of which was developed independently. Specifically, we address the DARPA-funded GALE information distillation Year 3 task of finding answers to the 5-Wh questions (who, what, when, where, and why) for each given sentence. The approach we take revolves around determining the best system using discriminative learning. In particular, we train support vector machines with a set of novel features that encode systems' capabilities of returning as many correct answers as possible. We analyze two combination strategies: one combines multiple systems at the granularity of sentences, and the other at the granularity of individual fields. Our experimental results indicate that the proposed features and combination strategies were able to improve the overall performance by 22% to 36% relative to a random selection, 16% to 35% relative to a majority voting scheme, and 15% to 23% relative to the best individual system.

KW - Question answering

KW - Systems for spoken language understanding

UR - http://www.scopus.com/inward/record.url?scp=70450176924&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=70450176924&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:70450176924

SP - 2703

EP - 2706

BT - Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH

ER -