An analysis of students' gaming behaviors in an intelligent tutoring system

Predictors and impacts

Kasia Muldner, Winslow Burleson, Brett Van De Sande, Kurt Vanlehn

Research output: Contribution to journalArticle

Abstract

Students who exploit properties of an instructional system to make progress while avoiding learning are said to be "gaming" the system. In order to investigate what causes gaming and how it impacts students, we analyzed log data from two Intelligent Tutoring Systems (ITS). The primary analyses focused on six college physics classes using the Andes ITS for homework and test preparation, starting with the research question: What is a better predictor of gaming, problem or student? To address this question, we developed a computational gaming detector for automatically labeling the Andes data, and applied several data mining techniques, including machine learning of Bayesian network parameters. Contrary to some prior findings, the analyses indicated that student was a better predictor of gaming than problem. This result was surprising, so we tested and confirmed it with log data from a second ITS (the Algebra Cognitive Tutor) and population (high school students). Given that student was more predictive of gaming than problem, subsequent analyses focused on how students gamed and in turn benefited (or not) from instructional features of the environment, as well as how gaming in general influenced problem solving and learning outcomes.

Original languageEnglish (US)
Pages (from-to)99-135
Number of pages37
JournalUser Modeling and User-Adapted Interaction
Volume21
Issue number1-2
DOIs
StatePublished - Apr 2011

Fingerprint

Intelligent systems
Students
student
learning
homework
Bayesian networks
tutor
Labeling
Algebra
Data mining
physics
Learning systems
Physics
Detectors
cause
school

Keywords

  • Bayesian network parameter learning
  • Educational data mining
  • Gaming
  • Utility of hints

ASJC Scopus subject areas

  • Human-Computer Interaction
  • Computer Science Applications
  • Education

Cite this

An analysis of students' gaming behaviors in an intelligent tutoring system : Predictors and impacts. / Muldner, Kasia; Burleson, Winslow; Van De Sande, Brett; Vanlehn, Kurt.

In: User Modeling and User-Adapted Interaction, Vol. 21, No. 1-2, 04.2011, p. 99-135.

Research output: Contribution to journalArticle

@article{ff9800c413e54d40ba0184505adf8499,
title = "An analysis of students' gaming behaviors in an intelligent tutoring system: Predictors and impacts",
abstract = "Students who exploit properties of an instructional system to make progress while avoiding learning are said to be {"}gaming{"} the system. In order to investigate what causes gaming and how it impacts students, we analyzed log data from two Intelligent Tutoring Systems (ITS). The primary analyses focused on six college physics classes using the Andes ITS for homework and test preparation, starting with the research question: What is a better predictor of gaming, problem or student? To address this question, we developed a computational gaming detector for automatically labeling the Andes data, and applied several data mining techniques, including machine learning of Bayesian network parameters. Contrary to some prior findings, the analyses indicated that student was a better predictor of gaming than problem. This result was surprising, so we tested and confirmed it with log data from a second ITS (the Algebra Cognitive Tutor) and population (high school students). Given that student was more predictive of gaming than problem, subsequent analyses focused on how students gamed and in turn benefited (or not) from instructional features of the environment, as well as how gaming in general influenced problem solving and learning outcomes.",
keywords = "Bayesian network parameter learning, Educational data mining, Gaming, Utility of hints",
author = "Kasia Muldner and Winslow Burleson and {Van De Sande}, Brett and Kurt Vanlehn",
year = "2011",
month = "4",
doi = "10.1007/s11257-010-9086-0",
language = "English (US)",
volume = "21",
pages = "99--135",
journal = "User Modelling and User-Adapted Interaction",
issn = "0924-1868",
publisher = "Springer Netherlands",
number = "1-2",

}

TY - JOUR

T1 - An analysis of students' gaming behaviors in an intelligent tutoring system

T2 - Predictors and impacts

AU - Muldner, Kasia

AU - Burleson, Winslow

AU - Van De Sande, Brett

AU - Vanlehn, Kurt

PY - 2011/4

Y1 - 2011/4

N2 - Students who exploit properties of an instructional system to make progress while avoiding learning are said to be "gaming" the system. In order to investigate what causes gaming and how it impacts students, we analyzed log data from two Intelligent Tutoring Systems (ITS). The primary analyses focused on six college physics classes using the Andes ITS for homework and test preparation, starting with the research question: What is a better predictor of gaming, problem or student? To address this question, we developed a computational gaming detector for automatically labeling the Andes data, and applied several data mining techniques, including machine learning of Bayesian network parameters. Contrary to some prior findings, the analyses indicated that student was a better predictor of gaming than problem. This result was surprising, so we tested and confirmed it with log data from a second ITS (the Algebra Cognitive Tutor) and population (high school students). Given that student was more predictive of gaming than problem, subsequent analyses focused on how students gamed and in turn benefited (or not) from instructional features of the environment, as well as how gaming in general influenced problem solving and learning outcomes.

AB - Students who exploit properties of an instructional system to make progress while avoiding learning are said to be "gaming" the system. In order to investigate what causes gaming and how it impacts students, we analyzed log data from two Intelligent Tutoring Systems (ITS). The primary analyses focused on six college physics classes using the Andes ITS for homework and test preparation, starting with the research question: What is a better predictor of gaming, problem or student? To address this question, we developed a computational gaming detector for automatically labeling the Andes data, and applied several data mining techniques, including machine learning of Bayesian network parameters. Contrary to some prior findings, the analyses indicated that student was a better predictor of gaming than problem. This result was surprising, so we tested and confirmed it with log data from a second ITS (the Algebra Cognitive Tutor) and population (high school students). Given that student was more predictive of gaming than problem, subsequent analyses focused on how students gamed and in turn benefited (or not) from instructional features of the environment, as well as how gaming in general influenced problem solving and learning outcomes.

KW - Bayesian network parameter learning

KW - Educational data mining

KW - Gaming

KW - Utility of hints

UR - http://www.scopus.com/inward/record.url?scp=79955795644&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=79955795644&partnerID=8YFLogxK

U2 - 10.1007/s11257-010-9086-0

DO - 10.1007/s11257-010-9086-0

M3 - Article

VL - 21

SP - 99

EP - 135

JO - User Modelling and User-Adapted Interaction

JF - User Modelling and User-Adapted Interaction

SN - 0924-1868

IS - 1-2

ER -