Evaluation of text coherence for electronic essay scoring systems

@article{Miltsakaki2004EvaluationOT,
  title={Evaluation of text coherence for electronic essay scoring systems},
  author={Eleni Miltsakaki and Karen Kukich},
  journal={Natural Language Engineering},
  year={2004},
  volume={10},
  pages={25 - 55}
}
Existing software systems for automated essay scoring can provide NLP researchers with opportunities to test certain theoretical hypotheses, including some derived from Centering Theory. In this study we employ the Educational Testing Service's e-rater essay scoring system to examine whether local discourse coherence, as defined by a measure of Centering Theory's Rough-Shift transitions, might be a significant contributor to the evaluation of essays. Rough-Shifts within students' paragraphs… Expand
Mining Domain Knowledge for Coherence Assessment of Students Proposal Drafts
TLDR
A coherence analyzer is presented, which employs latent semantic analysis (LSA) to mine existing corpora to further assess new drafts and uncovering a consistent interrelationship, according to methodology authors. Expand
Automated assessment of English-learner writing
TLDR
This thesis argues that an approach which does not rely on task-dependent components or data, and directly assesses learner English, can produce results as good as promptspecific models, and presents a user interface that visualises and uncovers the ‘marking criteria’ represented in AA models. Expand
Automated Essay Scoring for Swedish
We present the first system developed for automated grading of high school essays written in Swedish. The system uses standard text quality indicators and is able to compare vocabulary and grammar toExpand
Scoring for Swedish
We present the first system developed for automated grading of high school essays written in Swedish. The system uses standard text quality indicators and is able to compare vocabulary and grammar toExpand
Modeling coherence in ESOL learner texts
TLDR
This work presents the first systematic analysis of several methods for assessing coherence under the framework of automated assessment (AA) of learner free-text responses, and examines the predictive power of different coherence models by measuring the effect on performance when combined with an AA system that achieves competitive results. Expand
Modeling Organization in Student Essays
TLDR
A new annotated corpus is presented and heuristic-based and learning-based approaches to scoring essays along the organization dimension are proposed, utilizing techniques that involve sequence alignment, alignment kernels, and string kernels. Expand
Automated Detection of Local Coherence in Short Argumentative Essays Based on Centering Theory
TLDR
An automated method for assessing local coherence in short argumentative essays is described and it is suggested that focusing on nominal subject for detecting candidate concepts for a discourse segment's central concept is sufficient, which confirms previous findings. Expand
Modeling Thesis Clarity in Student Essays
TLDR
A new annotated corpus is presented and a learning- based approach to scoring essays along the thesis clarity dimension is proposed and a second learning-based approach to identifying what kinds of errors an essay has that may lower its thesis clarity score is proposed. Expand
Modelling text meta-properties in automated text scoring for non-native English writing
  • Meng Zhang
  • Computer Science
  • 2019
TLDR
This thesis showed that explicitly modelling these properties via frustratingly easy domain adaptation (FEDA) can positively affect model performance in some cases and showed that modelling multiple intra-exam properties together is better than modelling any single property individually or no property in four out of five test sets. Expand
Give Me More Feedback II: Annotating Thesis Strength and Related Attributes in Student Essays
TLDR
A scoring rubric is designed for scoring a core, yet unexplored dimension of persuasive essay quality, thesis strength, and annotate a corpus of essays with thesis strength scores to provide further feedback on why her essay receives a particular thesis strength score. Expand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 65 REFERENCES
Beyond Automated Essay Scoring
TLDR
Using computers to increase the understanding of the textual features and cognitive skills involved in creating and comprehending written text will have clear benefits and help develop more effective instructional materials for improving reading, writing, and other human communication abilities. Expand
The Computer Moves into Essay Grading: Updating the Ancient Test.
When it comes to reading essays and rating them, computers have not played much of a role to date. But that may change, according to Mr. Page and Ms. Petersen, because - for the first time ever - aExpand
Functional Centering - Grounding Referential Coherence in Information Structure
TLDR
A revision of the principles guiding the ordering of discourse entities in the forward-looking center list within the centering model is proposed, claiming that grammatical role criteria should be replaced by criteria that reflect the functional information structure of the utterances. Expand
Discourse Segmentation by Human and Automated Means
TLDR
The first part of this paper presents a method for empirically validating multitutterance units referred to as discourse segments, and reports highly significant results of segmentations performed by naive subjects, where a commonsense notion of speaker intention is the segmentation criterion. Expand
The Measurement of Textual Coherence with Latent Semantic Analysis.
TLDR
The approach for predicting coherence through reanalyzing sets of texts from 2 studies that manipulated the coherence of texts and assessed readers’ comprehension indicates that the method is able to predict the effect of text coherence on comprehension and is more effective than simple term‐term overlap measures. Expand
Specifying the Parameters of Centering Theory: a Corpus-Based Evaluation using Text from Application-Oriented Domains
TLDR
The main result is that trying to reduce the number of utterances without a backward-looking center results in an increased number of cases in which some discourse entity, but not the CB, gets pronominalized, and viceversa. Expand
An introduction to latent semantic analysis
TLDR
The adequacy of LSA's reflection of human knowledge has been established in a variety of ways, for example, its scores overlap those of humans on standard vocabulary and subject matter tests; it mimics human word sorting and category judgments; it simulates word‐word and passage‐word lexical priming data. Expand
A Corpus-based Investigation of Definite Description Use
TLDR
Questions are raised concerning the starategy of evaluating systems for definite description interpretation by comparing their results with a standardized annotation, and the great number of discourse-new definites and the presence of definites that did not seem to require a complete disambiguation. Expand
Centering, Anaphora Resolution, and Discourse Structure
TLDR
This chapter argues that the restriction of centering to operating within a discourse segment should be abandoned in order to integrate centering with a model of global discourse structure and shows how the cache model is easily integrated with the centering algorithm, and several types of data from naturally occurring discourses that support the proposed integrated model. Expand
Japanese Discourse and the Process of Centering
TLDR
It is shown that CENTERING provides constraints on when a ZERO can be interpreted as the ZERO TOPIC, and it is argued that while discourse interpretation is an inferential process, syntactic cues constrain this process. Expand
...
1
2
3
4
5
...