Towards automatic evaluation of learning object metadata quality

Xavier Ochoa, Erik Duval

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Thanks to recent developments on automatic generation of metadata and interoperability between repositories, the production, management and consumption of learning object metadata is vastly surpassing the human capacity to review or process these metadata. However, we need to make sure that the presence of some low quality metadata does not compromise the performance of services that rely on that information. Consequently, there is a need for automatic assessment of the quality of metadata, so that tools or users can be alerted about low quality instances. In this paper, we present several quality metrics for learning object metadata. We applied these metrics to a sample of records from a real repository and compared the results with the quality assessment given to the same records by a group of human reviewers. Through correlation and regression analysis, we found that one of the metrics, the text information content, could be used as a predictor of the human evaluation. While this metric is not a definitive measurement of the "real" quality of the metadata record, we present several ways in which it can be used. We also propose new research in other quality dimensions of the learning object metadata.

Original languageEnglish (US)
Title of host publicationAdvances in Conceptual Modeling
Subtitle of host publicationTheory and Practice - ER 2006 Workshops BP-UML, CoMoGIS, COSS, ECDM, OIS, QoIS, SemWAT, Proceedings
Pages372-381
Number of pages10
StatePublished - Dec 8 2006
Event25th International Conference on Conceptual Modeling, ER 2006 - Tucson, AZ, United States
Duration: Nov 6 2006Nov 9 2006

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume4231 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

Other25th International Conference on Conceptual Modeling, ER 2006
CountryUnited States
CityTucson, AZ
Period11/6/0611/9/06

Fingerprint

Learning Objects
Metadata
Learning
Evaluation
Metric
Repository
Production Management
Needs Assessment
Quality Assessment
Correlation Analysis
Information Content
Regression Analysis
Interoperability
Regression analysis
Predictors

ASJC Scopus subject areas

  • Computer Science(all)
  • Biochemistry, Genetics and Molecular Biology(all)
  • Theoretical Computer Science

Cite this

Ochoa, X., & Duval, E. (2006). Towards automatic evaluation of learning object metadata quality. In Advances in Conceptual Modeling: Theory and Practice - ER 2006 Workshops BP-UML, CoMoGIS, COSS, ECDM, OIS, QoIS, SemWAT, Proceedings (pp. 372-381). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 4231 LNCS).

Towards automatic evaluation of learning object metadata quality. / Ochoa, Xavier; Duval, Erik.

Advances in Conceptual Modeling: Theory and Practice - ER 2006 Workshops BP-UML, CoMoGIS, COSS, ECDM, OIS, QoIS, SemWAT, Proceedings. 2006. p. 372-381 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 4231 LNCS).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Ochoa, X & Duval, E 2006, Towards automatic evaluation of learning object metadata quality. in Advances in Conceptual Modeling: Theory and Practice - ER 2006 Workshops BP-UML, CoMoGIS, COSS, ECDM, OIS, QoIS, SemWAT, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 4231 LNCS, pp. 372-381, 25th International Conference on Conceptual Modeling, ER 2006, Tucson, AZ, United States, 11/6/06.
Ochoa X, Duval E. Towards automatic evaluation of learning object metadata quality. In Advances in Conceptual Modeling: Theory and Practice - ER 2006 Workshops BP-UML, CoMoGIS, COSS, ECDM, OIS, QoIS, SemWAT, Proceedings. 2006. p. 372-381. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
Ochoa, Xavier ; Duval, Erik. / Towards automatic evaluation of learning object metadata quality. Advances in Conceptual Modeling: Theory and Practice - ER 2006 Workshops BP-UML, CoMoGIS, COSS, ECDM, OIS, QoIS, SemWAT, Proceedings. 2006. pp. 372-381 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{0c15cc47105b4805b31a3538f26da6d3,
title = "Towards automatic evaluation of learning object metadata quality",
abstract = "Thanks to recent developments on automatic generation of metadata and interoperability between repositories, the production, management and consumption of learning object metadata is vastly surpassing the human capacity to review or process these metadata. However, we need to make sure that the presence of some low quality metadata does not compromise the performance of services that rely on that information. Consequently, there is a need for automatic assessment of the quality of metadata, so that tools or users can be alerted about low quality instances. In this paper, we present several quality metrics for learning object metadata. We applied these metrics to a sample of records from a real repository and compared the results with the quality assessment given to the same records by a group of human reviewers. Through correlation and regression analysis, we found that one of the metrics, the text information content, could be used as a predictor of the human evaluation. While this metric is not a definitive measurement of the {"}real{"} quality of the metadata record, we present several ways in which it can be used. We also propose new research in other quality dimensions of the learning object metadata.",
author = "Xavier Ochoa and Erik Duval",
year = "2006",
month = "12",
day = "8",
language = "English (US)",
isbn = "3540477039",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
pages = "372--381",
booktitle = "Advances in Conceptual Modeling",

}

TY - GEN

T1 - Towards automatic evaluation of learning object metadata quality

AU - Ochoa, Xavier

AU - Duval, Erik

PY - 2006/12/8

Y1 - 2006/12/8

N2 - Thanks to recent developments on automatic generation of metadata and interoperability between repositories, the production, management and consumption of learning object metadata is vastly surpassing the human capacity to review or process these metadata. However, we need to make sure that the presence of some low quality metadata does not compromise the performance of services that rely on that information. Consequently, there is a need for automatic assessment of the quality of metadata, so that tools or users can be alerted about low quality instances. In this paper, we present several quality metrics for learning object metadata. We applied these metrics to a sample of records from a real repository and compared the results with the quality assessment given to the same records by a group of human reviewers. Through correlation and regression analysis, we found that one of the metrics, the text information content, could be used as a predictor of the human evaluation. While this metric is not a definitive measurement of the "real" quality of the metadata record, we present several ways in which it can be used. We also propose new research in other quality dimensions of the learning object metadata.

AB - Thanks to recent developments on automatic generation of metadata and interoperability between repositories, the production, management and consumption of learning object metadata is vastly surpassing the human capacity to review or process these metadata. However, we need to make sure that the presence of some low quality metadata does not compromise the performance of services that rely on that information. Consequently, there is a need for automatic assessment of the quality of metadata, so that tools or users can be alerted about low quality instances. In this paper, we present several quality metrics for learning object metadata. We applied these metrics to a sample of records from a real repository and compared the results with the quality assessment given to the same records by a group of human reviewers. Through correlation and regression analysis, we found that one of the metrics, the text information content, could be used as a predictor of the human evaluation. While this metric is not a definitive measurement of the "real" quality of the metadata record, we present several ways in which it can be used. We also propose new research in other quality dimensions of the learning object metadata.

UR - http://www.scopus.com/inward/record.url?scp=33845189936&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=33845189936&partnerID=8YFLogxK

M3 - Conference contribution

SN - 3540477039

SN - 9783540477037

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 372

EP - 381

BT - Advances in Conceptual Modeling

ER -