Autonomously Generating Hints by Inferring Problem Solving Policies
@article{Piech2015AutonomouslyGH, title={Autonomously Generating Hints by Inferring Problem Solving Policies}, author={Chris Piech and Mehran Sahami and Jonathan Huang and Leonidas J. Guibas}, journal={Proceedings of the Second (2015) ACM Conference on Learning @ Scale}, year={2015} }
Exploring the whole sequence of steps a student takes to produce work, and the patterns that emerge from thousands of such sequences is fertile ground for a richer understanding of learning. In this paper we autonomously generate hints for the Code.org `Hour of Code,' (which is to the best of our knowledge the largest online course to date) using historical student data. We first develop a family of algorithms that can predict the way an expert teacher would encourage a student to make forward…
126 Citations
The Continuous Hint Factory - Providing Hints in Vast and Sparsely Populated Edit Distance Spaces
- Computer ScienceArXiv
- 2017
This contribution provides a mathematical framework for edit-based hint policies and proposes a novel hint policy to provide edit hints in vast and sparsely populated state spaces and demonstrates that the Continuous Hint Factory can predict more accurately what capable students would do compared to existing prediction schemes on two learning tasks.
Data-Driven Hint Generation in Vast Solution Spaces: a Self-Improving Python Programming Tutor
- Computer ScienceInternational Journal of Artificial Intelligence in Education
- 2015
The results show that ITAP is capable of producing hints for almost any given state after being given only a single reference solution, and that it can improve its performance by collecting data over time.
Automated Prediction of Novice Programmer Performance Using Programming Trajectories
- Computer ScienceAIED
- 2020
This work defines a supervised machine learning algorithm that takes as input students’ programming trajectories and predicts whether a student will successfully complete the next exercise and indicates that the model can accurately predict the future performance of the students.
Deep Knowledge Tracing On Programming Exercises
- Computer Science, EducationL@S
- 2017
This work feeds embedded program submissions into a recurrent neural network and train it on the task of predicting the student's success on the subsequent programming exercise, and reliably predicts future student performance.
Automated Data-Driven Hint Generation in Intelligent Tutoring Systems for Code-Writing: On the Road of Future Research
- Computer ScienceInt. J. Emerg. Technol. Learn.
- 2018
The goal in this paper is to review and classify analysis techniques that are requested to generate data-driven hints in ITSs for programming, and to identify the possible future directions in this research field.
Learning to Represent Student Knowledge on Programming Exercises Using Deep Learning
- Computer Science, EducationEDM
- 2017
This work feeds the embedded program submission sequence into a recurrent neural network and train it on two tasks of predicting the student’s future performance, and learns nuanced representations of a student's knowledge, exposes patterns about a student�'s learning behavior, and reliably predicts future student performance.
A Survey of Automated Programming Hint Generation: The HINTS Framework
- Computer ScienceACM Comput. Surv.
- 2022
All hint techniques can be understood as a series of simpler components with similar properties, and a simple framework for describing such techniques is presented, the Hint Iteration by Narrow-down and Transformation Steps (HINTS) framework.
A Classification of Data-Driven Hint Generation Techniques for Code-Writing Intelligent Tutoring Systems
- Computer Science, Education
- 2017
The goal in this paper is to review and classify analysis techniques that are requested to generate data-driven hints in ITSs for programming, and to propose several future research directions.
Generating Data-driven Hints for Open-ended Programming
- Computer ScienceEDM
- 2016
A new data-driven algorithm is presented, based on the Hint Factory, to generate hints for open-ended programming assignments, that can provide hints that successfully lead students to solutions from any state, help students achieve assignment objectives, and align with the student’s future solution.
Using Bugs in Student Code to Predict Need for Help
- Education2020 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC)
- 2020
This work uses information about the bugs present in student code to predict which students are more likely to abandon the puzzle or take too long in solving it, and envision that teachers could use these predictions to make decisions about whom they should help next, and how.
References
SHOWING 1-10 OF 25 REFERENCES
Toward Automatic Hint Generation for Logic Proof Tutoring Using Historical Student Data
- Computer ScienceIntelligent Tutoring Systems
- 2008
The feasibility of this approach to automatically generate hints for an intelligent tutor that learns is demonstrated by extracting MDPs from four semesters of student solutions in a logic proof tutor, and the probability that they will be able to generate hints at any point in a given problem is calculated.
Syntactic and Functional Variability of a Million Code Submissions in a Machine Learning MOOC
- Computer ScienceAIED Workshops
- 2013
The syntax and functional similarity of the submissions are mapped out in order to explore the variation in solutions in the first offering of Stanford's Machine Learning Massive Open-Access Online Course.
BlueFix: Using Crowd-Sourced Feedback to Support Programming Students in Error Diagnosis and Repair
- Computer ScienceICWL
- 2012
BlueFix is an online tool currently integrated into the BlueJ IDE which is designed to assist programming students with error diagnosis and repair and proposes a feedback algorithm based upon frameworks combined from the HCI and Pedagogical domains which can provide different students with dynamic levels of support based upon their compilation behaviour.
A Canonicalizing Model for Building Programming Tutors
- Computer ScienceITS
- 2012
This work has constructed a language-independent canonicalized model for programming solutions that allows for much greater overlap across different students than a basic text model, which enables more self-sustaining hint generation methods in programming tutors.
On automated grading of programming assignments in an academic institution
- Computer Science, EducationComput. Educ.
- 2003
Codewebs: scalable homework search for massive open online programming courses
- Computer ScienceWWW
- 2014
A method for decomposing online homework submissions into a vocabulary of "code phrases", and based on this vocabulary, a queryable index that allows for fast searches into the massive dataset of student homework submissions is designed.
Cognitive Tutor: Applied research in mathematics education
- EducationPsychonomic bulletin & review
- 2007
The theoretical background of this approach to cognitive models of mathematics, which have become a basis for middle- and high-school curricula, and how embedding a well specified theory in instructional software allows us to dynamically evaluate the effectiveness of the authors' instruction at a more detailed level than was previously possible.
An effective metacognitive strategy: learning by doing and explaining with a computer-based Cognitive Tutor
- Education, PsychologyCogn. Sci.
- 2002
On the Benefits of Seeking (and Avoiding) Help in Online Problem-Solving Environments
- Psychology
- 2014
It is suggested that novice learners may benefit from engaging in solution attempts before they can make sense of given assistance, as contrary to many help-seeking theories, avoiding help (and failing repeatedly) is associated with better learning.
Peer and self assessment in massive online classes
- EducationACM Trans. Comput. Hum. Interact.
- 2013
This article reports the experiences with two iterations of the first large online class to use peer and self-assessment, and finds that rubrics that use a parallel sentence structure, unambiguous wording, and well-specified dimensions have lower variance.