Overview of the SPMRL 2013 shared task: Cross-framework evaluation of parsing morphologically rich languages

Djamé Seddah, Reut Tsarfaty, Sandra Kübler, Marie Candito, Jinho D. Choi, Richárd Farkas, Jennifer Foster, Iakes Goenaga, Koldo Gojenola, Yoav Goldberg, Spence Green, Nizar Habash, Marco Kuhlmann, Wolfgang Maier, Joakim Nivre, Adam Przepiórkowski, Ryan Roth, Wolfgang Seeker, Yannick Versley, Veronika VinczeMarcin Woliński, Alina Wróblewska, Eric Villemonte De La Clérgerie

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

This paper reports on the first shared task on statistical parsing of morphologically rich languages (MRLs). The task features data sets from nine languages, each available both in constituency and dependency annotation. We report on the preparation of the data sets, on the proposed parsing scenarios, and on the evaluation metrics for parsing MRLs given different representation types. We present and analyze parsing results obtained by the task participants, and then provide an analysis and comparison of the parsers across languages and frameworks, reported for gold input as well as more realistic parsing scenarios.

Original languageEnglish (US)
Title of host publicationSPMRL 2013 - 4th Workshop on Statistical Parsing of Morphologically Rich Languages, Proceedings of the Workshop
PublisherAssociation for Computational Linguistics (ACL)
Pages146-182
Number of pages37
ISBN (Electronic)9781937284978
StatePublished - Jan 1 2013
Event4th Workshop on Statistical Parsing of Morphologically Rich Languages, SPMRL 2013 - Seattle, United States
Duration: Oct 18 2013 → …

Other

Other4th Workshop on Statistical Parsing of Morphologically Rich Languages, SPMRL 2013
CountryUnited States
CitySeattle
Period10/18/13 → …

Fingerprint

Gold
language
evaluation
scenario
gold
Evaluation
Language
Parsing
Scenarios

ASJC Scopus subject areas

  • Language and Linguistics
  • Computer Science Applications
  • Linguistics and Language

Cite this

Seddah, D., Tsarfaty, R., Kübler, S., Candito, M., Choi, J. D., Farkas, R., ... De La Clérgerie, E. V. (2013). Overview of the SPMRL 2013 shared task: Cross-framework evaluation of parsing morphologically rich languages. In SPMRL 2013 - 4th Workshop on Statistical Parsing of Morphologically Rich Languages, Proceedings of the Workshop (pp. 146-182). Association for Computational Linguistics (ACL).

Overview of the SPMRL 2013 shared task : Cross-framework evaluation of parsing morphologically rich languages. / Seddah, Djamé; Tsarfaty, Reut; Kübler, Sandra; Candito, Marie; Choi, Jinho D.; Farkas, Richárd; Foster, Jennifer; Goenaga, Iakes; Gojenola, Koldo; Goldberg, Yoav; Green, Spence; Habash, Nizar; Kuhlmann, Marco; Maier, Wolfgang; Nivre, Joakim; Przepiórkowski, Adam; Roth, Ryan; Seeker, Wolfgang; Versley, Yannick; Vincze, Veronika; Woliński, Marcin; Wróblewska, Alina; De La Clérgerie, Eric Villemonte.

SPMRL 2013 - 4th Workshop on Statistical Parsing of Morphologically Rich Languages, Proceedings of the Workshop. Association for Computational Linguistics (ACL), 2013. p. 146-182.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Seddah, D, Tsarfaty, R, Kübler, S, Candito, M, Choi, JD, Farkas, R, Foster, J, Goenaga, I, Gojenola, K, Goldberg, Y, Green, S, Habash, N, Kuhlmann, M, Maier, W, Nivre, J, Przepiórkowski, A, Roth, R, Seeker, W, Versley, Y, Vincze, V, Woliński, M, Wróblewska, A & De La Clérgerie, EV 2013, Overview of the SPMRL 2013 shared task: Cross-framework evaluation of parsing morphologically rich languages. in SPMRL 2013 - 4th Workshop on Statistical Parsing of Morphologically Rich Languages, Proceedings of the Workshop. Association for Computational Linguistics (ACL), pp. 146-182, 4th Workshop on Statistical Parsing of Morphologically Rich Languages, SPMRL 2013, Seattle, United States, 10/18/13.
Seddah D, Tsarfaty R, Kübler S, Candito M, Choi JD, Farkas R et al. Overview of the SPMRL 2013 shared task: Cross-framework evaluation of parsing morphologically rich languages. In SPMRL 2013 - 4th Workshop on Statistical Parsing of Morphologically Rich Languages, Proceedings of the Workshop. Association for Computational Linguistics (ACL). 2013. p. 146-182
Seddah, Djamé ; Tsarfaty, Reut ; Kübler, Sandra ; Candito, Marie ; Choi, Jinho D. ; Farkas, Richárd ; Foster, Jennifer ; Goenaga, Iakes ; Gojenola, Koldo ; Goldberg, Yoav ; Green, Spence ; Habash, Nizar ; Kuhlmann, Marco ; Maier, Wolfgang ; Nivre, Joakim ; Przepiórkowski, Adam ; Roth, Ryan ; Seeker, Wolfgang ; Versley, Yannick ; Vincze, Veronika ; Woliński, Marcin ; Wróblewska, Alina ; De La Clérgerie, Eric Villemonte. / Overview of the SPMRL 2013 shared task : Cross-framework evaluation of parsing morphologically rich languages. SPMRL 2013 - 4th Workshop on Statistical Parsing of Morphologically Rich Languages, Proceedings of the Workshop. Association for Computational Linguistics (ACL), 2013. pp. 146-182
@inproceedings{a9e76ece94b64620927d8894fe2c7351,
title = "Overview of the SPMRL 2013 shared task: Cross-framework evaluation of parsing morphologically rich languages",
abstract = "This paper reports on the first shared task on statistical parsing of morphologically rich languages (MRLs). The task features data sets from nine languages, each available both in constituency and dependency annotation. We report on the preparation of the data sets, on the proposed parsing scenarios, and on the evaluation metrics for parsing MRLs given different representation types. We present and analyze parsing results obtained by the task participants, and then provide an analysis and comparison of the parsers across languages and frameworks, reported for gold input as well as more realistic parsing scenarios.",
author = "Djam{\'e} Seddah and Reut Tsarfaty and Sandra K{\"u}bler and Marie Candito and Choi, {Jinho D.} and Rich{\'a}rd Farkas and Jennifer Foster and Iakes Goenaga and Koldo Gojenola and Yoav Goldberg and Spence Green and Nizar Habash and Marco Kuhlmann and Wolfgang Maier and Joakim Nivre and Adam Przepi{\'o}rkowski and Ryan Roth and Wolfgang Seeker and Yannick Versley and Veronika Vincze and Marcin Woliński and Alina Wr{\'o}blewska and {De La Cl{\'e}rgerie}, {Eric Villemonte}",
year = "2013",
month = "1",
day = "1",
language = "English (US)",
pages = "146--182",
booktitle = "SPMRL 2013 - 4th Workshop on Statistical Parsing of Morphologically Rich Languages, Proceedings of the Workshop",
publisher = "Association for Computational Linguistics (ACL)",

}

TY - GEN

T1 - Overview of the SPMRL 2013 shared task

T2 - Cross-framework evaluation of parsing morphologically rich languages

AU - Seddah, Djamé

AU - Tsarfaty, Reut

AU - Kübler, Sandra

AU - Candito, Marie

AU - Choi, Jinho D.

AU - Farkas, Richárd

AU - Foster, Jennifer

AU - Goenaga, Iakes

AU - Gojenola, Koldo

AU - Goldberg, Yoav

AU - Green, Spence

AU - Habash, Nizar

AU - Kuhlmann, Marco

AU - Maier, Wolfgang

AU - Nivre, Joakim

AU - Przepiórkowski, Adam

AU - Roth, Ryan

AU - Seeker, Wolfgang

AU - Versley, Yannick

AU - Vincze, Veronika

AU - Woliński, Marcin

AU - Wróblewska, Alina

AU - De La Clérgerie, Eric Villemonte

PY - 2013/1/1

Y1 - 2013/1/1

N2 - This paper reports on the first shared task on statistical parsing of morphologically rich languages (MRLs). The task features data sets from nine languages, each available both in constituency and dependency annotation. We report on the preparation of the data sets, on the proposed parsing scenarios, and on the evaluation metrics for parsing MRLs given different representation types. We present and analyze parsing results obtained by the task participants, and then provide an analysis and comparison of the parsers across languages and frameworks, reported for gold input as well as more realistic parsing scenarios.

AB - This paper reports on the first shared task on statistical parsing of morphologically rich languages (MRLs). The task features data sets from nine languages, each available both in constituency and dependency annotation. We report on the preparation of the data sets, on the proposed parsing scenarios, and on the evaluation metrics for parsing MRLs given different representation types. We present and analyze parsing results obtained by the task participants, and then provide an analysis and comparison of the parsers across languages and frameworks, reported for gold input as well as more realistic parsing scenarios.

UR - http://www.scopus.com/inward/record.url?scp=84926138689&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84926138689&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:84926138689

SP - 146

EP - 182

BT - SPMRL 2013 - 4th Workshop on Statistical Parsing of Morphologically Rich Languages, Proceedings of the Workshop

PB - Association for Computational Linguistics (ACL)

ER -