CoNLL 2017 shared task: Multilingual parsing from raw text to universal dependencies

Daniel Zeman, Martin Popel, Milan Straka, Jan Hajič, Joakim Nivre, Filip Ginter, Juhani Luotolahti, Sampo Pyysalo, Slav Petrov, Martin Potthast, Francis Tyers, Elena Badmaeva, Memduh Gökırmak, Anna Nedoluzhko, Silvie Cinková, Jan Hajič, Jaroslava Hlaváčová, Václava Kettnerová, Zdeňka Urešová, Jenna KanervaStina Ojala, Anna Missilä, Christopher Manning, Sebastian Schuster, Siva Reddy, Dima Taji, Nizar Habash, Herman Leung, Marie Catherine de Marneffe, Manuela Sanguinetti, Maria Simi, Hiroshi Kanayama, Valeria de Paiva, Kira Droganova, Héctor Martínez Alonso, Çağrı Çöltekin, Umut Sulubacak, Hans Uszkoreit, Vivien Macketanz, Aljoscha Burchardt, Kim Harris, Katrin Marheinecke, Georg Rehm, Tolga Kayadelen, Mohammed Attia, Ali Elkahky, Zhuoran Yu, Emily Pitler, Saran Lertpradit, Michael Mandl, Jesse Kirchner, Hector Fernandez Alcalde, Jana Strnadová, Esha Banerjee, Ruli Manurung, Antonio Stella, Atsuko Shimada, Sookyoung Kwak, Gustavo Mendonça, Tatiana Lando, Rattima Nitisaroj, Josie Li

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

The Conference on Computational Natural Language Learning (CoNLL) features a shared task, in which participants train and test their learning systems on the same data sets. In 2017, one of two tasks was devoted to learning dependency parsers for a large number of languages, in a real-world setting without any gold-standard annotation on input. All test sets followed a unified annotation scheme, namely that of Universal Dependencies. In this paper, we define the task and evaluation methodology, describe data preparation, report and analyze the main results, and provide a brief categorization of the different approaches of the participating systems.

Original languageEnglish (US)
Title of host publicationCoNLL 2017 - SIGNLL Conference on Computational Natural Language Learning, Proceedings of the CoNLL 2017 Shared Task
Subtitle of host publicationMultilingual Parsing from Raw Text to Universal Dependencies
PublisherAssociation for Computational Linguistics (ACL)
Pages1-19
Number of pages19
ISBN (Electronic)9781945626708
StatePublished - Jan 1 2017
Event2017 SIGNLL Conference on Computational Natural Language Learning- CoNLL Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, CoNLL 2017 - Vancouver, Canada
Duration: Aug 3 2017Aug 4 2017

Publication series

NameCoNLL 2017 - SIGNLL Conference on Computational Natural Language Learning, Proceedings of the CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies

Conference

Conference2017 SIGNLL Conference on Computational Natural Language Learning- CoNLL Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, CoNLL 2017
CountryCanada
CityVancouver
Period8/3/178/4/17

Fingerprint

Learning systems
Gold
language
data preparation
learning
gold standard
methodology
evaluation

ASJC Scopus subject areas

  • Artificial Intelligence
  • Human-Computer Interaction
  • Linguistics and Language

Cite this

Zeman, D., Popel, M., Straka, M., Hajič, J., Nivre, J., Ginter, F., ... Li, J. (2017). CoNLL 2017 shared task: Multilingual parsing from raw text to universal dependencies. In CoNLL 2017 - SIGNLL Conference on Computational Natural Language Learning, Proceedings of the CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies (pp. 1-19). (CoNLL 2017 - SIGNLL Conference on Computational Natural Language Learning, Proceedings of the CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies). Association for Computational Linguistics (ACL).

CoNLL 2017 shared task : Multilingual parsing from raw text to universal dependencies. / Zeman, Daniel; Popel, Martin; Straka, Milan; Hajič, Jan; Nivre, Joakim; Ginter, Filip; Luotolahti, Juhani; Pyysalo, Sampo; Petrov, Slav; Potthast, Martin; Tyers, Francis; Badmaeva, Elena; Gökırmak, Memduh; Nedoluzhko, Anna; Cinková, Silvie; Hajič, Jan; Hlaváčová, Jaroslava; Kettnerová, Václava; Urešová, Zdeňka; Kanerva, Jenna; Ojala, Stina; Missilä, Anna; Manning, Christopher; Schuster, Sebastian; Reddy, Siva; Taji, Dima; Habash, Nizar; Leung, Herman; de Marneffe, Marie Catherine; Sanguinetti, Manuela; Simi, Maria; Kanayama, Hiroshi; de Paiva, Valeria; Droganova, Kira; Alonso, Héctor Martínez; Çöltekin, Çağrı; Sulubacak, Umut; Uszkoreit, Hans; Macketanz, Vivien; Burchardt, Aljoscha; Harris, Kim; Marheinecke, Katrin; Rehm, Georg; Kayadelen, Tolga; Attia, Mohammed; Elkahky, Ali; Yu, Zhuoran; Pitler, Emily; Lertpradit, Saran; Mandl, Michael; Kirchner, Jesse; Alcalde, Hector Fernandez; Strnadová, Jana; Banerjee, Esha; Manurung, Ruli; Stella, Antonio; Shimada, Atsuko; Kwak, Sookyoung; Mendonça, Gustavo; Lando, Tatiana; Nitisaroj, Rattima; Li, Josie.

CoNLL 2017 - SIGNLL Conference on Computational Natural Language Learning, Proceedings of the CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies. Association for Computational Linguistics (ACL), 2017. p. 1-19 (CoNLL 2017 - SIGNLL Conference on Computational Natural Language Learning, Proceedings of the CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Zeman, D, Popel, M, Straka, M, Hajič, J, Nivre, J, Ginter, F, Luotolahti, J, Pyysalo, S, Petrov, S, Potthast, M, Tyers, F, Badmaeva, E, Gökırmak, M, Nedoluzhko, A, Cinková, S, Hajič, J, Hlaváčová, J, Kettnerová, V, Urešová, Z, Kanerva, J, Ojala, S, Missilä, A, Manning, C, Schuster, S, Reddy, S, Taji, D, Habash, N, Leung, H, de Marneffe, MC, Sanguinetti, M, Simi, M, Kanayama, H, de Paiva, V, Droganova, K, Alonso, HM, Çöltekin, Ç, Sulubacak, U, Uszkoreit, H, Macketanz, V, Burchardt, A, Harris, K, Marheinecke, K, Rehm, G, Kayadelen, T, Attia, M, Elkahky, A, Yu, Z, Pitler, E, Lertpradit, S, Mandl, M, Kirchner, J, Alcalde, HF, Strnadová, J, Banerjee, E, Manurung, R, Stella, A, Shimada, A, Kwak, S, Mendonça, G, Lando, T, Nitisaroj, R & Li, J 2017, CoNLL 2017 shared task: Multilingual parsing from raw text to universal dependencies. in CoNLL 2017 - SIGNLL Conference on Computational Natural Language Learning, Proceedings of the CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies. CoNLL 2017 - SIGNLL Conference on Computational Natural Language Learning, Proceedings of the CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, Association for Computational Linguistics (ACL), pp. 1-19, 2017 SIGNLL Conference on Computational Natural Language Learning- CoNLL Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, CoNLL 2017, Vancouver, Canada, 8/3/17.
Zeman D, Popel M, Straka M, Hajič J, Nivre J, Ginter F et al. CoNLL 2017 shared task: Multilingual parsing from raw text to universal dependencies. In CoNLL 2017 - SIGNLL Conference on Computational Natural Language Learning, Proceedings of the CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies. Association for Computational Linguistics (ACL). 2017. p. 1-19. (CoNLL 2017 - SIGNLL Conference on Computational Natural Language Learning, Proceedings of the CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies).
Zeman, Daniel ; Popel, Martin ; Straka, Milan ; Hajič, Jan ; Nivre, Joakim ; Ginter, Filip ; Luotolahti, Juhani ; Pyysalo, Sampo ; Petrov, Slav ; Potthast, Martin ; Tyers, Francis ; Badmaeva, Elena ; Gökırmak, Memduh ; Nedoluzhko, Anna ; Cinková, Silvie ; Hajič, Jan ; Hlaváčová, Jaroslava ; Kettnerová, Václava ; Urešová, Zdeňka ; Kanerva, Jenna ; Ojala, Stina ; Missilä, Anna ; Manning, Christopher ; Schuster, Sebastian ; Reddy, Siva ; Taji, Dima ; Habash, Nizar ; Leung, Herman ; de Marneffe, Marie Catherine ; Sanguinetti, Manuela ; Simi, Maria ; Kanayama, Hiroshi ; de Paiva, Valeria ; Droganova, Kira ; Alonso, Héctor Martínez ; Çöltekin, Çağrı ; Sulubacak, Umut ; Uszkoreit, Hans ; Macketanz, Vivien ; Burchardt, Aljoscha ; Harris, Kim ; Marheinecke, Katrin ; Rehm, Georg ; Kayadelen, Tolga ; Attia, Mohammed ; Elkahky, Ali ; Yu, Zhuoran ; Pitler, Emily ; Lertpradit, Saran ; Mandl, Michael ; Kirchner, Jesse ; Alcalde, Hector Fernandez ; Strnadová, Jana ; Banerjee, Esha ; Manurung, Ruli ; Stella, Antonio ; Shimada, Atsuko ; Kwak, Sookyoung ; Mendonça, Gustavo ; Lando, Tatiana ; Nitisaroj, Rattima ; Li, Josie. / CoNLL 2017 shared task : Multilingual parsing from raw text to universal dependencies. CoNLL 2017 - SIGNLL Conference on Computational Natural Language Learning, Proceedings of the CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies. Association for Computational Linguistics (ACL), 2017. pp. 1-19 (CoNLL 2017 - SIGNLL Conference on Computational Natural Language Learning, Proceedings of the CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies).
@inproceedings{4c634fa3a4214f3db747f03bbfd5a34e,
title = "CoNLL 2017 shared task: Multilingual parsing from raw text to universal dependencies",
abstract = "The Conference on Computational Natural Language Learning (CoNLL) features a shared task, in which participants train and test their learning systems on the same data sets. In 2017, one of two tasks was devoted to learning dependency parsers for a large number of languages, in a real-world setting without any gold-standard annotation on input. All test sets followed a unified annotation scheme, namely that of Universal Dependencies. In this paper, we define the task and evaluation methodology, describe data preparation, report and analyze the main results, and provide a brief categorization of the different approaches of the participating systems.",
author = "Daniel Zeman and Martin Popel and Milan Straka and Jan Hajič and Joakim Nivre and Filip Ginter and Juhani Luotolahti and Sampo Pyysalo and Slav Petrov and Martin Potthast and Francis Tyers and Elena Badmaeva and Memduh G{\"o}kırmak and Anna Nedoluzhko and Silvie Cinkov{\'a} and Jan Hajič and Jaroslava Hlav{\'a}čov{\'a} and V{\'a}clava Kettnerov{\'a} and Zdeňka Urešov{\'a} and Jenna Kanerva and Stina Ojala and Anna Missil{\"a} and Christopher Manning and Sebastian Schuster and Siva Reddy and Dima Taji and Nizar Habash and Herman Leung and {de Marneffe}, {Marie Catherine} and Manuela Sanguinetti and Maria Simi and Hiroshi Kanayama and {de Paiva}, Valeria and Kira Droganova and Alonso, {H{\'e}ctor Mart{\'i}nez} and {\cC}ağrı {\cC}{\"o}ltekin and Umut Sulubacak and Hans Uszkoreit and Vivien Macketanz and Aljoscha Burchardt and Kim Harris and Katrin Marheinecke and Georg Rehm and Tolga Kayadelen and Mohammed Attia and Ali Elkahky and Zhuoran Yu and Emily Pitler and Saran Lertpradit and Michael Mandl and Jesse Kirchner and Alcalde, {Hector Fernandez} and Jana Strnadov{\'a} and Esha Banerjee and Ruli Manurung and Antonio Stella and Atsuko Shimada and Sookyoung Kwak and Gustavo Mendon{\cc}a and Tatiana Lando and Rattima Nitisaroj and Josie Li",
year = "2017",
month = "1",
day = "1",
language = "English (US)",
series = "CoNLL 2017 - SIGNLL Conference on Computational Natural Language Learning, Proceedings of the CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies",
publisher = "Association for Computational Linguistics (ACL)",
pages = "1--19",
booktitle = "CoNLL 2017 - SIGNLL Conference on Computational Natural Language Learning, Proceedings of the CoNLL 2017 Shared Task",

}

TY - GEN

T1 - CoNLL 2017 shared task

T2 - Multilingual parsing from raw text to universal dependencies

AU - Zeman, Daniel

AU - Popel, Martin

AU - Straka, Milan

AU - Hajič, Jan

AU - Nivre, Joakim

AU - Ginter, Filip

AU - Luotolahti, Juhani

AU - Pyysalo, Sampo

AU - Petrov, Slav

AU - Potthast, Martin

AU - Tyers, Francis

AU - Badmaeva, Elena

AU - Gökırmak, Memduh

AU - Nedoluzhko, Anna

AU - Cinková, Silvie

AU - Hajič, Jan

AU - Hlaváčová, Jaroslava

AU - Kettnerová, Václava

AU - Urešová, Zdeňka

AU - Kanerva, Jenna

AU - Ojala, Stina

AU - Missilä, Anna

AU - Manning, Christopher

AU - Schuster, Sebastian

AU - Reddy, Siva

AU - Taji, Dima

AU - Habash, Nizar

AU - Leung, Herman

AU - de Marneffe, Marie Catherine

AU - Sanguinetti, Manuela

AU - Simi, Maria

AU - Kanayama, Hiroshi

AU - de Paiva, Valeria

AU - Droganova, Kira

AU - Alonso, Héctor Martínez

AU - Çöltekin, Çağrı

AU - Sulubacak, Umut

AU - Uszkoreit, Hans

AU - Macketanz, Vivien

AU - Burchardt, Aljoscha

AU - Harris, Kim

AU - Marheinecke, Katrin

AU - Rehm, Georg

AU - Kayadelen, Tolga

AU - Attia, Mohammed

AU - Elkahky, Ali

AU - Yu, Zhuoran

AU - Pitler, Emily

AU - Lertpradit, Saran

AU - Mandl, Michael

AU - Kirchner, Jesse

AU - Alcalde, Hector Fernandez

AU - Strnadová, Jana

AU - Banerjee, Esha

AU - Manurung, Ruli

AU - Stella, Antonio

AU - Shimada, Atsuko

AU - Kwak, Sookyoung

AU - Mendonça, Gustavo

AU - Lando, Tatiana

AU - Nitisaroj, Rattima

AU - Li, Josie

PY - 2017/1/1

Y1 - 2017/1/1

N2 - The Conference on Computational Natural Language Learning (CoNLL) features a shared task, in which participants train and test their learning systems on the same data sets. In 2017, one of two tasks was devoted to learning dependency parsers for a large number of languages, in a real-world setting without any gold-standard annotation on input. All test sets followed a unified annotation scheme, namely that of Universal Dependencies. In this paper, we define the task and evaluation methodology, describe data preparation, report and analyze the main results, and provide a brief categorization of the different approaches of the participating systems.

AB - The Conference on Computational Natural Language Learning (CoNLL) features a shared task, in which participants train and test their learning systems on the same data sets. In 2017, one of two tasks was devoted to learning dependency parsers for a large number of languages, in a real-world setting without any gold-standard annotation on input. All test sets followed a unified annotation scheme, namely that of Universal Dependencies. In this paper, we define the task and evaluation methodology, describe data preparation, report and analyze the main results, and provide a brief categorization of the different approaches of the participating systems.

UR - http://www.scopus.com/inward/record.url?scp=85073123106&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85073123106&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:85073123106

T3 - CoNLL 2017 - SIGNLL Conference on Computational Natural Language Learning, Proceedings of the CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies

SP - 1

EP - 19

BT - CoNLL 2017 - SIGNLL Conference on Computational Natural Language Learning, Proceedings of the CoNLL 2017 Shared Task

PB - Association for Computational Linguistics (ACL)

ER -