• Corpus ID: 14468405

Generating Data-driven Hints for Open-ended Programming

@inproceedings{Price2016GeneratingDH,
  title={Generating Data-driven Hints for Open-ended Programming},
  author={Thomas W. Price and Yihuan Dong and Tiffany Barnes},
  booktitle={EDM},
  year={2016}
}
Intelligent Tutoring Systems (ITSs) have shown success in the domain of programming, in part by providing customized hints and feedback to students. However, many popular novice programming environments still lack these intelligent features. This is due in part to their use of open-ended programming assignments, which are difficult to support with existing hint generation techniques. In this paper, we present a new data-driven algorithm, based on the Hint Factory, to generate hints for these… 

Figures and Tables from this paper

A Classification of Data-Driven Hint Generation Techniques for Code-Writing Intelligent Tutoring Systems
  • H. Bui
  • Computer Science, Education
  • 2017
TLDR
The goal in this paper is to review and classify analysis techniques that are requested to generate data-driven hints in ITSs for programming, and to propose several future research directions.
Automated Data-Driven Hint Generation in Intelligent Tutoring Systems for Code-Writing: On the Road of Future Research
TLDR
The goal in this paper is to review and classify analysis techniques that are requested to generate data-driven hints in ITSs for programming, and to identify the possible future directions in this research field.
iSnap: Towards Intelligent Tutoring in Novice Programming Environments
TLDR
Results from a pilot study of iSnap are shared, indicating that students are generally willing to use hints and that hints can create positive outcomes, and some key challenges encountered in the pilot study are highlighted.
Evaluation of a Data-driven Feedback Algorithm for Open-ended Programming
TLDR
A novel, data-driven algorithm for generating feedback for students on open-ended programming problems, which goes beyond next-step hints, annotating a student’s whole program with suggested edits, including code that should be moved or reordered.
Automated Data-Driven Hint Generation for Learning Programming
TLDR
Intelligent tutoring systems can provide personalized feedback to students automatically, but they can take large amounts of time and expert knowledge to build, especially when determining how to give students hints.
A Survey of Automated Programming Hint Generation: The HINTS Framework
TLDR
All hint techniques can be understood as a series of simpler components with similar properties, and a simple framework for describing such techniques is presented, the Hint Iteration by Narrow-down and Transformation Steps (HINTS) framework.
The Continuous Hint Factory - Providing Hints in Vast and Sparsely Populated Edit Distance Spaces
TLDR
This contribution provides a mathematical framework for edit-based hint policies and proposes a novel hint policy to provide edit hints in vast and sparsely populated state spaces and demonstrates that the Continuous Hint Factory can predict more accurately what capable students would do compared to existing prediction schemes on two learning tasks.
Guiding Next-Step Hint Generation Using Automated Tests
TLDR
Catnip is presented, the first nextstep hint generation approach for Scratch, which extends existing data-driven hint generation approaches with automated testing and allows the hints to be better individualized for the chosen solution path.
Model-Driven Synthesis for Programming Tutors
TLDR
This work proposes to investigate how to overcome the completeness problem of giving automated feedback to students by using program synthesis, which is used to generate correct solutions that closely match a student program.
The Impact of Data Quantity and Source on the Quality of Data-Driven Hints for Programming
TLDR
It is found that with student training data, hint quality stops improving after 15–20 training solutions and can decrease with additional data, and that student data outperforms a single expert solution but that a comprehensive set of expert solutions generally performs best.
...
...

References

SHOWING 1-10 OF 28 REFERENCES
Data-Driven Hint Generation in Vast Solution Spaces: a Self-Improving Python Programming Tutor
TLDR
The results show that ITAP is capable of producing hints for almost any given state after being given only a single reference solution, and that it can improve its performance by collecting data over time.
An Exploration of Data-Driven Hint Generation in an Open-Ended Programming Problem
TLDR
Current strategies for generating data-driven hints for programming problems are reviewed and their applicability to larger, more open-ended problems, with multiple, loosely ordered goals is examined.
Toward Automatic Hint Generation for Logic Proof Tutoring Using Historical Student Data
TLDR
The feasibility of this approach to automatically generate hints for an intelligent tutor that learns is demonstrated by extracting MDPs from four semesters of student solutions in a logic proof tutor, and the probability that they will be able to generate hints at any point in a given problem is calculated.
Generating Hints for Programming Problems Using Intermediate Output
TLDR
In the context of the educational programming game, BOTS, it is found that worldstates require less prior data to generate hints in a majority of cases, without sacrificing quality or interpretability.
Experimental Evaluation of Automatic Hint Generation for a Logic Tutor
TLDR
This work augmented the Deep Thought logic tutor with a Hint Factory that generates data-driven, context-specific hints for an existing computer aided instructional tool, and shows that hints help students persist in a deductive logic proofs tutor.
Data driven automatic feedback generation in the iList intelligent tutoring system
TLDR
The evaluation study showed that iList is effective in helping students learn; students liked working with the system; and the feedback generated by the most sophisticated versions of the system is helpful in keeping students on the right path.
Autonomously Generating Hints by Inferring Problem Solving Policies
TLDR
This paper autonomously generate hints for the Code.org `Hour of Code,' (which is to the best of the authors' knowledge the largest online course to date) using historical student data, and discovers that this statistic is highly predictive of a student's future success.
Automatic Generation of Programming Feedback; A Data-Driven Approach
TLDR
A data-driven approach for automatic feedback generation which utilizes the program solution space to predict where a student is located within the set of many possible learning progressions and what their next steps should be is proposed.
A Canonicalizing Model for Building Programming Tutors
TLDR
This work has constructed a language-independent canonicalized model for programming solutions that allows for much greater overlap across different students than a basic text model, which enables more self-sustaining hint generation methods in programming tutors.
An Automated Framework for Recommending Program Elements to Novices (N)
TLDR
A general framework for recognizing the desired target for partially-written code and recommending a reliable series of edits to transform the input program into the target solution, based on graph matching and tree edit algorithms is presented.
...
...