Measuring contribution in collaborative writing: An adaptive NMF topic modelling approach

Johnny Torres, Alberto Jimenez, Sixto García, Enrique Peláez, Xavier Ochoa

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

In universities worldwide, instructors may spend a significant amount of time reviewing homework and group projects submitted by their students. Web-based technologies, like Google Docs, have provided a platform for students to write documents collaboratively. Currently, those platforms provide limited information on the individual contribution made by each student. Previous studies have focused on the quantitative aspects of individuals' contribution in collaborative writing, while the quality aspect has received less attention. In this paper, we propose a new model to measure not only quantitative input but also the quality of the content that has been contributed to a document written collaboratively in Spanish language. Based on topics-modeling techniques, we use an adaptive non-negative matrix factorization (NMF) model to extract topics from the content of the document, and grade higher students making those contributions. Using Google documents submitted by students to the academic system of our university as part of their projects, experimental results show that compared to other baseline methods such as edits or words count, our model provide a better approximation to the scores given by human reviewers. Therefore, our model can be used as part of an automatic grading subsystem within the academic system, to provide a baseline score of students' contribution in collaborative documents. This will allow instructors to reduce their workload associated with revision and grading of documents and focus their time on more relevant tasks.

Original languageEnglish (US)
Title of host publication2017 4th International Conference on eDemocracy and eGovernment, ICEDEG 2017
EditorsLuis Teran, Luis Teran, Andreas Meier
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages63-70
Number of pages8
ISBN (Electronic)9781509048304
DOIs
StatePublished - Jun 29 2017
Event4th International Conference on eDemocracy and eGovernment, ICEDEG 2017 - Quito, Ecuador
Duration: Apr 19 2017Apr 21 2017

Other

Other4th International Conference on eDemocracy and eGovernment, ICEDEG 2017
CountryEcuador
CityQuito
Period4/19/174/21/17

Fingerprint

Factorization
Students
student
grading
search engine
instructor
project group
Spanish language
university
homework
subsystem
workload
Collaborative writing
Modeling
Matrix factorization
Grading
Google

Keywords

  • collaborative writing
  • Education technology
  • topic modeling

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Information Systems
  • Sociology and Political Science
  • Communication
  • Public Administration
  • Information Systems and Management
  • Education

Cite this

Torres, J., Jimenez, A., García, S., Peláez, E., & Ochoa, X. (2017). Measuring contribution in collaborative writing: An adaptive NMF topic modelling approach. In L. Teran, L. Teran, & A. Meier (Eds.), 2017 4th International Conference on eDemocracy and eGovernment, ICEDEG 2017 (pp. 63-70). [7962514] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICEDEG.2017.7962514

Measuring contribution in collaborative writing : An adaptive NMF topic modelling approach. / Torres, Johnny; Jimenez, Alberto; García, Sixto; Peláez, Enrique; Ochoa, Xavier.

2017 4th International Conference on eDemocracy and eGovernment, ICEDEG 2017. ed. / Luis Teran; Luis Teran; Andreas Meier. Institute of Electrical and Electronics Engineers Inc., 2017. p. 63-70 7962514.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Torres, J, Jimenez, A, García, S, Peláez, E & Ochoa, X 2017, Measuring contribution in collaborative writing: An adaptive NMF topic modelling approach. in L Teran, L Teran & A Meier (eds), 2017 4th International Conference on eDemocracy and eGovernment, ICEDEG 2017., 7962514, Institute of Electrical and Electronics Engineers Inc., pp. 63-70, 4th International Conference on eDemocracy and eGovernment, ICEDEG 2017, Quito, Ecuador, 4/19/17. https://doi.org/10.1109/ICEDEG.2017.7962514
Torres J, Jimenez A, García S, Peláez E, Ochoa X. Measuring contribution in collaborative writing: An adaptive NMF topic modelling approach. In Teran L, Teran L, Meier A, editors, 2017 4th International Conference on eDemocracy and eGovernment, ICEDEG 2017. Institute of Electrical and Electronics Engineers Inc. 2017. p. 63-70. 7962514 https://doi.org/10.1109/ICEDEG.2017.7962514
Torres, Johnny ; Jimenez, Alberto ; García, Sixto ; Peláez, Enrique ; Ochoa, Xavier. / Measuring contribution in collaborative writing : An adaptive NMF topic modelling approach. 2017 4th International Conference on eDemocracy and eGovernment, ICEDEG 2017. editor / Luis Teran ; Luis Teran ; Andreas Meier. Institute of Electrical and Electronics Engineers Inc., 2017. pp. 63-70
@inproceedings{f5d7a3e7476047ca9e17857f5a37c0cc,
title = "Measuring contribution in collaborative writing: An adaptive NMF topic modelling approach",
abstract = "In universities worldwide, instructors may spend a significant amount of time reviewing homework and group projects submitted by their students. Web-based technologies, like Google Docs, have provided a platform for students to write documents collaboratively. Currently, those platforms provide limited information on the individual contribution made by each student. Previous studies have focused on the quantitative aspects of individuals' contribution in collaborative writing, while the quality aspect has received less attention. In this paper, we propose a new model to measure not only quantitative input but also the quality of the content that has been contributed to a document written collaboratively in Spanish language. Based on topics-modeling techniques, we use an adaptive non-negative matrix factorization (NMF) model to extract topics from the content of the document, and grade higher students making those contributions. Using Google documents submitted by students to the academic system of our university as part of their projects, experimental results show that compared to other baseline methods such as edits or words count, our model provide a better approximation to the scores given by human reviewers. Therefore, our model can be used as part of an automatic grading subsystem within the academic system, to provide a baseline score of students' contribution in collaborative documents. This will allow instructors to reduce their workload associated with revision and grading of documents and focus their time on more relevant tasks.",
keywords = "collaborative writing, Education technology, topic modeling",
author = "Johnny Torres and Alberto Jimenez and Sixto Garc{\'i}a and Enrique Pel{\'a}ez and Xavier Ochoa",
year = "2017",
month = "6",
day = "29",
doi = "10.1109/ICEDEG.2017.7962514",
language = "English (US)",
pages = "63--70",
editor = "Luis Teran and Luis Teran and Andreas Meier",
booktitle = "2017 4th International Conference on eDemocracy and eGovernment, ICEDEG 2017",
publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

TY - GEN

T1 - Measuring contribution in collaborative writing

T2 - An adaptive NMF topic modelling approach

AU - Torres, Johnny

AU - Jimenez, Alberto

AU - García, Sixto

AU - Peláez, Enrique

AU - Ochoa, Xavier

PY - 2017/6/29

Y1 - 2017/6/29

N2 - In universities worldwide, instructors may spend a significant amount of time reviewing homework and group projects submitted by their students. Web-based technologies, like Google Docs, have provided a platform for students to write documents collaboratively. Currently, those platforms provide limited information on the individual contribution made by each student. Previous studies have focused on the quantitative aspects of individuals' contribution in collaborative writing, while the quality aspect has received less attention. In this paper, we propose a new model to measure not only quantitative input but also the quality of the content that has been contributed to a document written collaboratively in Spanish language. Based on topics-modeling techniques, we use an adaptive non-negative matrix factorization (NMF) model to extract topics from the content of the document, and grade higher students making those contributions. Using Google documents submitted by students to the academic system of our university as part of their projects, experimental results show that compared to other baseline methods such as edits or words count, our model provide a better approximation to the scores given by human reviewers. Therefore, our model can be used as part of an automatic grading subsystem within the academic system, to provide a baseline score of students' contribution in collaborative documents. This will allow instructors to reduce their workload associated with revision and grading of documents and focus their time on more relevant tasks.

AB - In universities worldwide, instructors may spend a significant amount of time reviewing homework and group projects submitted by their students. Web-based technologies, like Google Docs, have provided a platform for students to write documents collaboratively. Currently, those platforms provide limited information on the individual contribution made by each student. Previous studies have focused on the quantitative aspects of individuals' contribution in collaborative writing, while the quality aspect has received less attention. In this paper, we propose a new model to measure not only quantitative input but also the quality of the content that has been contributed to a document written collaboratively in Spanish language. Based on topics-modeling techniques, we use an adaptive non-negative matrix factorization (NMF) model to extract topics from the content of the document, and grade higher students making those contributions. Using Google documents submitted by students to the academic system of our university as part of their projects, experimental results show that compared to other baseline methods such as edits or words count, our model provide a better approximation to the scores given by human reviewers. Therefore, our model can be used as part of an automatic grading subsystem within the academic system, to provide a baseline score of students' contribution in collaborative documents. This will allow instructors to reduce their workload associated with revision and grading of documents and focus their time on more relevant tasks.

KW - collaborative writing

KW - Education technology

KW - topic modeling

UR - http://www.scopus.com/inward/record.url?scp=85026836526&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85026836526&partnerID=8YFLogxK

U2 - 10.1109/ICEDEG.2017.7962514

DO - 10.1109/ICEDEG.2017.7962514

M3 - Conference contribution

AN - SCOPUS:85026836526

SP - 63

EP - 70

BT - 2017 4th International Conference on eDemocracy and eGovernment, ICEDEG 2017

A2 - Teran, Luis

A2 - Teran, Luis

A2 - Meier, Andreas

PB - Institute of Electrical and Electronics Engineers Inc.

ER -