Automating Hint Generation with Solution Space Path Construction

@inproceedings{Rivers2014AutomatingHG,
  title={Automating Hint Generation with Solution Space Path Construction},
  author={Kelly Rivers and K. Koedinger},
  booktitle={Intelligent Tutoring Systems},
  year={2014}
}
Developing intelligent tutoring systems from student solution data is a promising approach to facilitating more widespread application of tutors. In principle, tutor feedback can be generated by matching student solution attempts to stored intermediate solution states, and next-step hints can be generated by finding a path from a student's current state to a correct solution state. However, exact matching of states and paths does not work for many domains, like programming, where the number of… 
View via Publisher
krivers.net
58 Citations
Highly Influential Citations
7
Background Citations
36
Methods Citations
14
Results Citations
2

58 Citations

The Continuous Hint Factory - Providing Hints in Vast and Sparsely Populated Edit Distance Spaces
TLDR
This contribution provides a mathematical framework for edit-based hint policies and proposes a novel hint policy to provide edit hints in vast and sparsely populated state spaces and demonstrates that the Continuous Hint Factory can predict more accurately what capable students would do compared to existing prediction schemes on two learning tasks.
Autonomously Generating Hints by Inferring Problem Solving Policies
TLDR
This paper autonomously generate hints for the Code.org `Hour of Code,' (which is to the best of the authors' knowledge the largest online course to date) using historical student data, and discovers that this statistic is highly predictive of a student's future success.
Data-Driven Hint Generation in Vast Solution Spaces: a Self-Improving Python Programming Tutor
TLDR
The results show that ITAP is capable of producing hints for almost any given state after being given only a single reference solution, and that it can improve its performance by collecting data over time.
Using the Hint Factory to Compare Model-Based Tutoring Systems
TLDR
It is argued that the state-space analysis can be used to better understand students’ problem-solving strategies and can beused to highlight the impact of different design decisions.
A Survey of Automated Programming Hint Generation: The HINTS Framework
TLDR
All hint techniques can be understood as a series of simpler components with similar properties, and a simple framework for describing such techniques is presented, the Hint Iteration by Narrow-down and Transformation Steps (HINTS) framework.
High-Coverage Hint Generation for Massive Courses: Do Automated Hints Help CS1 Students?
TLDR
A robust hint generation system that extends the coverage of the mutation-based approach using two complementary techniques and shows that hints contributed to students' progress while still encouraging the students to solve problems by themselves.
Authoring Tutors with Complex Solutions: A Comparative Analysis of Example Tracing and SimStudent
TLDR
A framework for understanding solution space complexity is presented and the abilities of Example Tracing and SimStudent for authoring problems in an experimental design tutor are analyzed, finding that both non-programming approaches support authoring of this complex problem.
Current State and Next Steps on Automated Hints for Students Learning to Code
TLDR
A design science approach is taken to define an ATS based on APR that attempts to address the identified challenges and suggests a three-step roadmap.
High Coverage Hint Generation for Massive Courses by Sumukh Sridhara Research Project
TLDR
A robust hint generation system that extends the coverage of the mutation-based approach using two complementary techniques and shows that hints contributed to students’ progress while still encouraging the students to solve problems by themselves is described.
Ask-Elle: an Adaptable Programming Tutor for Haskell Giving Automated Feedback
TLDR
The design of a tutor that combines the incremental development of different solutions in various forms to a programming exercise with automated feedback and teacher-specified programming exercises, solutions, and properties is designed.
...
...

References

SHOWING 1-8 OF 8 REFERENCES
Toward Automatic Hint Generation for Logic Proof Tutoring Using Historical Student Data
TLDR
The feasibility of this approach to automatically generate hints for an intelligent tutor that learns is demonstrated by extracting MDPs from four semesters of student solutions in a logic proof tutor, and the probability that they will be able to generate hints at any point in a given problem is calculated.
Feedback Provision Strategies in Intelligent Tutoring Systems Based on Clustered Solution Spaces
TLDR
This paper proposes the provision of feedback based on solution spaces which are automatically clustered by machine learning techniques operating on sets of student solutions and validated in an expert evaluation with a data set from a programming course.
Domain-Independent Proximity Measures in Intelligent Tutoring Systems
TLDR
It is proposed to identify and match meaningful contextual components in the solutions of intelligent tutoring systems based on the ample premise that solutions can be represented as formal graphs, and present first evaluation results for artificial as well as real student solutions.
The Behavior of Tutoring Systems
  • K. VanLehn
  • Computer Science
    Int. J. Artif. Intell. Educ.
  • 2006
TLDR
Although tutoring systems differ widely in their task domains, user interfaces, software structures, knowledge bases, etc., their behaviors are in fact quite similar.
A Canonicalizing Model for Building Programming Tutors
TLDR
This work has constructed a language-independent canonicalized model for programming solutions that allows for much greater overlap across different students than a basic text model, which enables more self-sustaining hint generation methods in programming tutors.
Syntactic and Functional Variability of a Million Code Submissions in a Machine Learning MOOC
TLDR
The syntax and functional similarity of the submissions are mapped out in order to explore the variation in solutions in the first offering of Stanford's Machine Learning Massive Open-Access Online Course.
Using learning analytics to assess students' behavior in open-ended programming tasks
TLDR
This paper logs hundreds of snapshots of students' code during a programming assignment, and employs different quantitative techniques to extract students' behaviors and categorize them in terms of programming experience.
DomainIndependent Proximity Measures in Intelligent Tutoring Systems
  • In Proceedings of the 6th International Conference on Educational Data Mining (EDM) (pp. 334-335)
  • 2013