Who, What, When, Where, Why? Comparing multiple approaches to the cross-lingual 5W task

Kristen Parton, Kathleen R. McKeown, Bob Coyne, Mona T. Diab, Ralph Grishman, Dilek Hakkani-Tür, Mary Harper, Heng Ji, Weiyun Ma, Adam Meyers, Sara Stolbach, Ang Sun, Gokhan Tur, Wei Xu, Sibel Yaman

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Cross-lingual tasks are especially difficult due to the compounding effect of errors in language processing and errors in machine translation (MT). In this paper, we present an error analysis of a new cross-lingual task: the 5W task, a sentence-level understanding task which seeks to return the English 5W's (Who, What, When, Where and Why) corresponding to a Chinese sentence. We analyze systems that we developed, identifying specific problems in language processing and MT that cause errors. The best cross-lingual 5W system was still 19% worse than the best monolingual 5W system, which shows that MT significantly degrades sentence-level understanding. Neither source-language nor targetlanguage analysis was able to circumvent problems in MT, although each approach had advantages relative to the other. A detailed error analysis across multiple systems suggests directions for future research on the problem.

Original languageEnglish (US)
Title of host publicationACL-IJCNLP 2009 - Joint Conf. of the 47th Annual Meeting of the Association for Computational Linguistics and 4th Int. Joint Conf. on Natural Language Processing of the AFNLP, Proceedings of the Conf.
Pages423-431
Number of pages9
StatePublished - 2009
EventJoint Conference of the 47th Annual Meeting of the Association for Computational Linguistics and 4th International Joint Conference on Natural Language Processing of the AFNLP, ACL-IJCNLP 2009 - Suntec, Singapore
Duration: Aug 2 2009Aug 7 2009

Other

OtherJoint Conference of the 47th Annual Meeting of the Association for Computational Linguistics and 4th International Joint Conference on Natural Language Processing of the AFNLP, ACL-IJCNLP 2009
CountrySingapore
CitySuntec
Period8/2/098/7/09

Fingerprint

language
Machine Translation
cause
Language Processing
Error Analysis
Compounding
Source Language
Causes

ASJC Scopus subject areas

  • Language and Linguistics
  • Linguistics and Language

Cite this

Parton, K., McKeown, K. R., Coyne, B., Diab, M. T., Grishman, R., Hakkani-Tür, D., ... Yaman, S. (2009). Who, What, When, Where, Why? Comparing multiple approaches to the cross-lingual 5W task. In ACL-IJCNLP 2009 - Joint Conf. of the 47th Annual Meeting of the Association for Computational Linguistics and 4th Int. Joint Conf. on Natural Language Processing of the AFNLP, Proceedings of the Conf. (pp. 423-431)

Who, What, When, Where, Why? Comparing multiple approaches to the cross-lingual 5W task. / Parton, Kristen; McKeown, Kathleen R.; Coyne, Bob; Diab, Mona T.; Grishman, Ralph; Hakkani-Tür, Dilek; Harper, Mary; Ji, Heng; Ma, Weiyun; Meyers, Adam; Stolbach, Sara; Sun, Ang; Tur, Gokhan; Xu, Wei; Yaman, Sibel.

ACL-IJCNLP 2009 - Joint Conf. of the 47th Annual Meeting of the Association for Computational Linguistics and 4th Int. Joint Conf. on Natural Language Processing of the AFNLP, Proceedings of the Conf.. 2009. p. 423-431.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Parton, K, McKeown, KR, Coyne, B, Diab, MT, Grishman, R, Hakkani-Tür, D, Harper, M, Ji, H, Ma, W, Meyers, A, Stolbach, S, Sun, A, Tur, G, Xu, W & Yaman, S 2009, Who, What, When, Where, Why? Comparing multiple approaches to the cross-lingual 5W task. in ACL-IJCNLP 2009 - Joint Conf. of the 47th Annual Meeting of the Association for Computational Linguistics and 4th Int. Joint Conf. on Natural Language Processing of the AFNLP, Proceedings of the Conf.. pp. 423-431, Joint Conference of the 47th Annual Meeting of the Association for Computational Linguistics and 4th International Joint Conference on Natural Language Processing of the AFNLP, ACL-IJCNLP 2009, Suntec, Singapore, 8/2/09.
Parton K, McKeown KR, Coyne B, Diab MT, Grishman R, Hakkani-Tür D et al. Who, What, When, Where, Why? Comparing multiple approaches to the cross-lingual 5W task. In ACL-IJCNLP 2009 - Joint Conf. of the 47th Annual Meeting of the Association for Computational Linguistics and 4th Int. Joint Conf. on Natural Language Processing of the AFNLP, Proceedings of the Conf.. 2009. p. 423-431
Parton, Kristen ; McKeown, Kathleen R. ; Coyne, Bob ; Diab, Mona T. ; Grishman, Ralph ; Hakkani-Tür, Dilek ; Harper, Mary ; Ji, Heng ; Ma, Weiyun ; Meyers, Adam ; Stolbach, Sara ; Sun, Ang ; Tur, Gokhan ; Xu, Wei ; Yaman, Sibel. / Who, What, When, Where, Why? Comparing multiple approaches to the cross-lingual 5W task. ACL-IJCNLP 2009 - Joint Conf. of the 47th Annual Meeting of the Association for Computational Linguistics and 4th Int. Joint Conf. on Natural Language Processing of the AFNLP, Proceedings of the Conf.. 2009. pp. 423-431
@inproceedings{0de58c97f6cb4aed9d271697a391b65e,
title = "Who, What, When, Where, Why? Comparing multiple approaches to the cross-lingual 5W task",
abstract = "Cross-lingual tasks are especially difficult due to the compounding effect of errors in language processing and errors in machine translation (MT). In this paper, we present an error analysis of a new cross-lingual task: the 5W task, a sentence-level understanding task which seeks to return the English 5W's (Who, What, When, Where and Why) corresponding to a Chinese sentence. We analyze systems that we developed, identifying specific problems in language processing and MT that cause errors. The best cross-lingual 5W system was still 19{\%} worse than the best monolingual 5W system, which shows that MT significantly degrades sentence-level understanding. Neither source-language nor targetlanguage analysis was able to circumvent problems in MT, although each approach had advantages relative to the other. A detailed error analysis across multiple systems suggests directions for future research on the problem.",
author = "Kristen Parton and McKeown, {Kathleen R.} and Bob Coyne and Diab, {Mona T.} and Ralph Grishman and Dilek Hakkani-T{\"u}r and Mary Harper and Heng Ji and Weiyun Ma and Adam Meyers and Sara Stolbach and Ang Sun and Gokhan Tur and Wei Xu and Sibel Yaman",
year = "2009",
language = "English (US)",
isbn = "9781617382581",
pages = "423--431",
booktitle = "ACL-IJCNLP 2009 - Joint Conf. of the 47th Annual Meeting of the Association for Computational Linguistics and 4th Int. Joint Conf. on Natural Language Processing of the AFNLP, Proceedings of the Conf.",

}

TY - GEN

T1 - Who, What, When, Where, Why? Comparing multiple approaches to the cross-lingual 5W task

AU - Parton, Kristen

AU - McKeown, Kathleen R.

AU - Coyne, Bob

AU - Diab, Mona T.

AU - Grishman, Ralph

AU - Hakkani-Tür, Dilek

AU - Harper, Mary

AU - Ji, Heng

AU - Ma, Weiyun

AU - Meyers, Adam

AU - Stolbach, Sara

AU - Sun, Ang

AU - Tur, Gokhan

AU - Xu, Wei

AU - Yaman, Sibel

PY - 2009

Y1 - 2009

N2 - Cross-lingual tasks are especially difficult due to the compounding effect of errors in language processing and errors in machine translation (MT). In this paper, we present an error analysis of a new cross-lingual task: the 5W task, a sentence-level understanding task which seeks to return the English 5W's (Who, What, When, Where and Why) corresponding to a Chinese sentence. We analyze systems that we developed, identifying specific problems in language processing and MT that cause errors. The best cross-lingual 5W system was still 19% worse than the best monolingual 5W system, which shows that MT significantly degrades sentence-level understanding. Neither source-language nor targetlanguage analysis was able to circumvent problems in MT, although each approach had advantages relative to the other. A detailed error analysis across multiple systems suggests directions for future research on the problem.

AB - Cross-lingual tasks are especially difficult due to the compounding effect of errors in language processing and errors in machine translation (MT). In this paper, we present an error analysis of a new cross-lingual task: the 5W task, a sentence-level understanding task which seeks to return the English 5W's (Who, What, When, Where and Why) corresponding to a Chinese sentence. We analyze systems that we developed, identifying specific problems in language processing and MT that cause errors. The best cross-lingual 5W system was still 19% worse than the best monolingual 5W system, which shows that MT significantly degrades sentence-level understanding. Neither source-language nor targetlanguage analysis was able to circumvent problems in MT, although each approach had advantages relative to the other. A detailed error analysis across multiple systems suggests directions for future research on the problem.

UR - http://www.scopus.com/inward/record.url?scp=80053388676&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=80053388676&partnerID=8YFLogxK

M3 - Conference contribution

SN - 9781617382581

SP - 423

EP - 431

BT - ACL-IJCNLP 2009 - Joint Conf. of the 47th Annual Meeting of the Association for Computational Linguistics and 4th Int. Joint Conf. on Natural Language Processing of the AFNLP, Proceedings of the Conf.

ER -