Utility evaluation of cross-document information extraction

Heng Ji, Zheng Chen, Jonathan Feldman, Antonio Gonzalez, Ralph Grishman, Vivek Upadhyay

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We describe a utility evaluation to determine whether cross-document information extraction (IE) techniques measurably improve user performance in news summary writing. Two groups of subjects were asked to perform the same time-restricted summary writing tasks, reading news under different conditions: with no IE results at all, with traditional singledocument IE results, and with cross-document IE results. Our results show that, in comparison to using source documents only, the quality of summary reports assembled using IE results, especially from cross-document IE, was significantly better and user satisfaction was higher. We also compare the impact of different user groups on the results.

Original languageEnglish (US)
Title of host publicationNAACL HLT 2010 - Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, Proceedings of the Main Conference
Pages285-288
Number of pages4
StatePublished - 2010
Event2010 Human Language Technologies Conference ofthe North American Chapter of the Association for Computational Linguistics, NAACL HLT 2010 - Los Angeles, CA, United States
Duration: Jun 2 2010Jun 4 2010

Other

Other2010 Human Language Technologies Conference ofthe North American Chapter of the Association for Computational Linguistics, NAACL HLT 2010
CountryUnited States
CityLos Angeles, CA
Period6/2/106/4/10

Fingerprint

evaluation
news
Evaluation
Information Extraction
Group
performance
Summary
News

ASJC Scopus subject areas

  • Language and Linguistics
  • Linguistics and Language

Cite this

Ji, H., Chen, Z., Feldman, J., Gonzalez, A., Grishman, R., & Upadhyay, V. (2010). Utility evaluation of cross-document information extraction. In NAACL HLT 2010 - Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, Proceedings of the Main Conference (pp. 285-288)

Utility evaluation of cross-document information extraction. / Ji, Heng; Chen, Zheng; Feldman, Jonathan; Gonzalez, Antonio; Grishman, Ralph; Upadhyay, Vivek.

NAACL HLT 2010 - Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, Proceedings of the Main Conference. 2010. p. 285-288.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Ji, H, Chen, Z, Feldman, J, Gonzalez, A, Grishman, R & Upadhyay, V 2010, Utility evaluation of cross-document information extraction. in NAACL HLT 2010 - Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, Proceedings of the Main Conference. pp. 285-288, 2010 Human Language Technologies Conference ofthe North American Chapter of the Association for Computational Linguistics, NAACL HLT 2010, Los Angeles, CA, United States, 6/2/10.
Ji H, Chen Z, Feldman J, Gonzalez A, Grishman R, Upadhyay V. Utility evaluation of cross-document information extraction. In NAACL HLT 2010 - Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, Proceedings of the Main Conference. 2010. p. 285-288
Ji, Heng ; Chen, Zheng ; Feldman, Jonathan ; Gonzalez, Antonio ; Grishman, Ralph ; Upadhyay, Vivek. / Utility evaluation of cross-document information extraction. NAACL HLT 2010 - Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, Proceedings of the Main Conference. 2010. pp. 285-288
@inproceedings{fcdff15a1c3b4b3cb15d3c276783a5a1,
title = "Utility evaluation of cross-document information extraction",
abstract = "We describe a utility evaluation to determine whether cross-document information extraction (IE) techniques measurably improve user performance in news summary writing. Two groups of subjects were asked to perform the same time-restricted summary writing tasks, reading news under different conditions: with no IE results at all, with traditional singledocument IE results, and with cross-document IE results. Our results show that, in comparison to using source documents only, the quality of summary reports assembled using IE results, especially from cross-document IE, was significantly better and user satisfaction was higher. We also compare the impact of different user groups on the results.",
author = "Heng Ji and Zheng Chen and Jonathan Feldman and Antonio Gonzalez and Ralph Grishman and Vivek Upadhyay",
year = "2010",
language = "English (US)",
isbn = "1932432655",
pages = "285--288",
booktitle = "NAACL HLT 2010 - Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, Proceedings of the Main Conference",

}

TY - GEN

T1 - Utility evaluation of cross-document information extraction

AU - Ji, Heng

AU - Chen, Zheng

AU - Feldman, Jonathan

AU - Gonzalez, Antonio

AU - Grishman, Ralph

AU - Upadhyay, Vivek

PY - 2010

Y1 - 2010

N2 - We describe a utility evaluation to determine whether cross-document information extraction (IE) techniques measurably improve user performance in news summary writing. Two groups of subjects were asked to perform the same time-restricted summary writing tasks, reading news under different conditions: with no IE results at all, with traditional singledocument IE results, and with cross-document IE results. Our results show that, in comparison to using source documents only, the quality of summary reports assembled using IE results, especially from cross-document IE, was significantly better and user satisfaction was higher. We also compare the impact of different user groups on the results.

AB - We describe a utility evaluation to determine whether cross-document information extraction (IE) techniques measurably improve user performance in news summary writing. Two groups of subjects were asked to perform the same time-restricted summary writing tasks, reading news under different conditions: with no IE results at all, with traditional singledocument IE results, and with cross-document IE results. Our results show that, in comparison to using source documents only, the quality of summary reports assembled using IE results, especially from cross-document IE, was significantly better and user satisfaction was higher. We also compare the impact of different user groups on the results.

UR - http://www.scopus.com/inward/record.url?scp=84863337594&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84863337594&partnerID=8YFLogxK

M3 - Conference contribution

SN - 1932432655

SN - 9781932432657

SP - 285

EP - 288

BT - NAACL HLT 2010 - Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, Proceedings of the Main Conference

ER -