# Toward Automatic Hint Generation for Logic Proof Tutoring Using Historical Student Data

@inproceedings{Barnes2008TowardAH, title={Toward Automatic Hint Generation for Logic Proof Tutoring Using Historical Student Data}, author={Tiffany Barnes and John C. Stamper}, booktitle={Intelligent Tutoring Systems}, year={2008} }

We have proposed a novel application of Markov decision processes (MDPs), a reinforcement learning technique, to automatically generate hints for an intelligent tutor that learns. We demonstrate the feasibility of this approach by extracting MDPs from four semesters of student solutions in a logic proof tutor, and calculating the probability that we will be able to generate hints at any point in a given problem. Our results indicate that extracted MDPs and our proposed hint-generating functionsâ€¦Â

## 137 Citations

A pilot study on logic proof tutoring using hints generated from historical student data

- Computer ScienceEDM
- 2008

The results of the pilot study using Deep Thought with the Hint Factory demonstrate that hints generated from historical data can support students in writing logic proofs.

Unsupervised MDP Value Selection for Automating ITS Capabilities.

- Computer ScienceEDM 2009
- 2009

A novel method for assigning prior values to student work that depends only on frequency of occurrence for the component steps is proposed, and the results show that the utility metric outperforms a classic MDP solution in selecting hints in logic.

An Improved Data-Driven Hint Selection Algorithm for Probability Tutors

- Computer ScienceEDM
- 2015

A novel hint-selection algorithm for coherent derivational domains, such as probability, which addresses this problem by searching a frontier of viable, partially matching student states by providing higher value hints to students in unknown states.

Experimental Evaluation of Automatic Hint Generation for a Logic Tutor

- Computer ScienceInt. J. Artif. Intell. Educ.
- 2013

This work augmented the Deep Thought logic tutor with a Hint Factory that generates data-driven, context-specific hints for an existing computer aided instructional tool, and shows that hints help students persist in a deductive logic proofs tutor.

Enhancing the Automatic Generation of Hints with Expert Seeding

- Computer ScienceInt. J. Artif. Intell. Educ.
- 2011

This paper describes the use of expert sample solutions to "seed" the hint generation process, and shows that just a few expert solutions give significant coverage (over 50%) for hints.

Autonomously Generating Hints by Inferring Problem Solving Policies

- Computer ScienceL@S
- 2015

This paper autonomously generate hints for the Code.org `Hour of Code,' (which is to the best of the authors' knowledge the largest online course to date) using historical student data, and discovers that this statistic is highly predictive of a student's future success.

Automatic Generation of Deductive Logic Proof Problems

- Computer ScienceAIED
- 2011

This work aims to automatically generate proofs for deductive logic proofs in such a way that fulfills the parameters set by the instructors, while using the progress recorded to generate further questions specific to the individual student.

Effects of Automatically Generated Hints on Time in a Logic Tutor

- Computer ScienceAIED Workshops
- 2013

This work explores the effects of using automatically generated hints in Deep Thought, a propositional logic tutor, and finds a consistent trend in which students without hints spent more time on problems when compared to students that were provided hints.

Generating Data-driven Hints for Open-ended Programming

- Computer ScienceEDM
- 2016

A new data-driven algorithm is presented, based on the Hint Factory, to generate hints for open-ended programming assignments, that can provide hints that successfully lead students to solutions from any state, help students achieve assignment objectives, and align with the studentâ€™s future solution.

Enhancing the Automatic Generation of Hints with Expert Seeding

- Computer ScienceIntelligent Tutoring Systems
- 2010

This paper describes the use of expert sample solutions to â€śseedâ€ť the hint generation process and shows that just a few expert solutions give significant coverage for hints.

## References

SHOWING 1-10 OF 18 REFERENCES

Toward the extraction of production rules for solving logic proofs

- Computer Science
- 2008

This work proposes a novel application of Markov decision processes (MDPs), a reinforcement learning technique, to automatically extract production rules for an intelligent tutor that learns, and demonstrates the feasibility of this approach by extracting MDPs from student solutions in a logic proof tutor, and using these to analyze and visualize student work.

Towards an Intelligent Tutoring System for Propositional Proof Construction

- Education
- 2008

Two project goals are addressed here: identifying at-risk students at an early stage in the semester, and generating a visual representation of student proof efforts as a step toward understanding those efforts.

Opening the Door to Non-programmers: Authoring Intelligent Tutor Behavior by Demonstration

- Computer ScienceIntelligent Tutoring Systems
- 2004

A method and set of software tools that ease the process of cognitive task analysis and tutor development by allowing the author to demonstrate, instead of programming, the behavior of an intelligent tutor without requiring AI programming are described.

ADVISOR: A Machine Learning Architecture for Intelligent Tutor Construction

- Computer Science, EducationAAAI/IAAI
- 2000

ADVISOR, a two-agent machine learning architecture for intelligent tutoring systems (ITS), is constructed to centralize the reasoning of an ITS into a single component to allow customization of teaching goals and to simplify improving the ITS.

Bootstrapping Novice Data: Semi-Automated Tutor Authoring Using Student Log Files

- Computer Science
- 2004

A potentially powerful way to aid in the authoring of intelligent tutoring systems is to directly leverage student interaction log data to directly construct an initial tutoring system model.

A Comparative Analysis of Cognitive Tutoring and Constraint-Based Modeling

- Computer ScienceUser Modeling
- 2003

This paper compares Cognitive Tutoring to Constraint-Based Modeling (CBM) and argues the differences are often more apparent than real: for specific domains one approach may be favoured over the other, making them viable complementary methods for supporting learning.

Authoring Intelligent Tutoring Systems: An analysis of the state of the art

- Computer Science
- 1999

An in-depth summary and analysis of the research and development state of the art for intelligent tutoring system (ITS) authoring systems and the major unknowns and bottlenecks to having widespread use of ITS authoring tools.

Effective Feedback Content for Tutoring Complex Skills

- PsychologyHum. Comput. Interact.
- 1990

Results of a study using the Geometry Tutor for building geometry proofs found that feedback about the goal structure of geometry problems led to better performance than Feedback about the reasons for error or than simply being told that an error had occurred.

Intelligent Tutoring Systems

- Computer ScienceScience
- 1985

Computer tutors based on a set of pedagogical principles derived from the ACT theory of cognition have been developed for teaching students to do proofs in geometry and to write computer programs in the language LISP.

Reinforcement Learning: An Introduction

- Computer ScienceIEEE Transactions on Neural Networks
- 2005

This book provides a clear and simple account of the key ideas and algorithms of reinforcement learning, which ranges from the history of the field's intellectual foundations to the most recent developments and applications.