DeepStealth: Game-Based Learning Stealth Assessment With Deep Neural Networks

  title={DeepStealth: Game-Based Learning Stealth Assessment With Deep Neural Networks},
  author={Wookhee Min and Megan Hardy Frankosky and Bradford W. Mott and Jonathan P. Rowe and A. Smith and Eric N. Wiebe and Kristy Elizabeth Boyer and James C. Lester},
  journal={IEEE Transactions on Learning Technologies},
A distinctive feature of game-based learning environments is their capacity for enabling stealth assessment. Stealth assessment analyzes a stream of fine-grained student interaction data from a game-based learning environment to dynamically draw inferences about students’ competencies through evidence-centered design. In evidence-centered design, evidence models have been traditionally designed using statistical rules authored by domain experts that are encoded using Bayesian networks. This… 

Figures and Tables from this paper

Enhancing Student Competency Models for Game-Based Learning with a Hybrid Stealth Assessment Framework

A hybrid data-driven approach to stealth assessment for predicting students’ mastery of concepts through interactions with a gamebased learning environment for introductory genetics and the results suggest that the hybrid stealth assessment framework outperforms individual models and holds significant potential for predicting student competencies.

Multimodal Multi-Task Stealth Assessment for Reflection-Enriched Game-Based Learning

A gated recurrent unit-based multi-task stealth assessment framework that takes as input multimodal data streams to jointly predict post-test scores and written reflection depth scores shows significant promise for supporting student learning and improved reflection.

Predictive Student Modeling in Game-Based Learning Environments with Word Embedding Representations of Reflection

A predictive student modeling framework that leverages natural language responses to in-game reflection prompts to predict student learning outcomes in a game-based learning environment for middle school microbiology, CRYSTAL ISLAND is presented.

Multimodal learning analytics for game-based learning

A distinctive feature of game-based learning environments is their capacity to create learning experiences that are both effective and engaging. Recent advances in sensor-based technologies such as

LogCF: Deep Collaborative Filtering with Process Data for Enhanced Learning Outcome Modeling

Leveraging the collaborative filtering approach used in recommender systems, a general framework of deep learning-based collaborative filtering with process data for enhanced learning outcome modeling is developed, which is named LogCF, capable of modeling learner- and item-skill associations as well as predicting learners' item responses.

Predictive Student Modeling in Educational Games with Multi-Task Learning

This work reformulates predictive student modeling as a multi-task learning problem, modeling questions from student test data as distinct “tasks” and shows that multi- Task stacked LSTMs with residual connections significantly outperform baseline models that do not use the multi- task formulation.

Utilizing Game Analytics to Inform and Validate Digital Game-based Assessment with Evidence-centered Game Design: A Case Study

The purpose of this case study is to demonstrate how to utilize machine learning approaches to analyze student process data for validating and informing digital game-based assessments (DGBAs) with an

Maximizing learning without sacrificing the fun: Stealth assessment, adaptivity and learning supports in educational games

Results showed that the stealth assessment estimates of physics understanding were valid—significantly correlating with the external physics test scores; there was no significant effect of game-level delivery method on students' learning; and physics animations were the most effective (among eight supports tested) in predicting both learning outcome and in-game performance.

I NTELLI B LOX : A Toolkit for Integrating Block-Based Programming into Game-Based Learning Environments

I NTELLI B LOX is presented, a Blockly-inspired toolkit for the Unity cross-platform game engine that enables learners to create block-based programs within immersive game-based learning environments.

Position: IntelliBlox: A Toolkit for Integrating Block-Based Programming into Game-Based Learning Environments

IntelliBlox is presented, a Blockly-inspired toolkit for the Unity cross-platform game engine that enables learners to create block-based programs within immersive game-based learning environments.



DeepStealth: Leveraging Deep Learning Models for Stealth Assessment in Game-Based Learning Environments

A framework for stealth assessment that leverages deep learning, a family of machine learning methods that utilize deep artificial neural networks, to infer student competencies in a game-based learning environment for middle grade computational thinking, Engage is presented.

Inducing Stealth Assessors from Game Interaction Data

A long short-term memory network (LSTM)-based stealth assessment framework that takes as input an observed sequence of raw game-based learning environment interaction data along with external pre-learning measures to infer students’ post-competencies and indicates that the LSTM-based approach holds significant promise for evidence modeling in stealth assessment.

Deep Learning-Based Goal Recognition in Open-Ended Digital Games

A goal recognition framework based on stacked denoising autoencoders, a variant of deep learning, which significantly outperforms the previous state-of-the-art goal recognition approach based on Markov logic networks.

A Generalized Multidimensional Evaluation Framework for Player Goal Recognition

GOALIE, a multidimensional framework for evaluating player goal recognition models, is introduced and the results suggest that GOALIE effectively captures goal recognition behaviors that are key to next-generation player modeling.

Interactive Narrative Personalization with Deep Reinforcement Learning

This paper presents a deep RL-based interactive narrative generation framework that leverages synthetic data produced by a bipartite simulated player model and investigates the deep RL framework’s performance with an educational interactive narrative, CRYSTAL ISLAND.

A Data-Driven Approach for Inferring Student Proficiency from Game Activity Logs

A novel data analysis pipeline, Student Proficiency Inferrer from Game data (SPRING), that allows modeling game playing behavior in educational games and can predict math assessments accurately on withheld test data.

Deep Knowledge Tracing

The utility of using Recurrent Neural Networks to model student learning and the learned model can be used for intelligent curriculum design and allows straightforward interpretation and discovery of structure in student tasks are explored.

Enhancing Student Models in Game-based Learning with Facial Expression Recognition

The study found that models based on individual facial action coding units are more effective than composite emotion models, and suggests that introducing facial expression tracking can improve the accuracy of student models, both for predicting student learning gains and also for predictingStudent engagement.

Greedy Layer-Wise Training of Deep Networks

These experiments confirm the hypothesis that the greedy layer-wise unsupervised training strategy mostly helps the optimization, by initializing weights in a region near a good local minimum, giving rise to internal distributed representations that are high-level abstractions of the input, bringing better generalization.

Personalizing Embedded Assessment Sequences in Narrative-Centered Learning Environments: A Collaborative Filtering Approach

This paper compares two model-based collaborative filtering methods, including probabilistic principal component analysis and non-negative matrix factorization (NMF), to a memory-based baseline model, k-nearest neighbor, and suggests that PPCA provides the most accurate predictions on average, but NMF provides a better balance between accuracy and run-time efficiency.