• Corpus ID: 53359038

Fairness of Automated Essay Scoring of GMAT ® AWA

@inproceedings{Guo2009FairnessOA,
  title={Fairness of Automated Essay Scoring of GMAT {\textregistered} AWA},
  author={Fanmin Guo},
  year={2009}
}
This study investigates the fairness of the automated essay scoring from the Analytical Writing Assessment to six subpopulation groups of Graduate Management Admission Test ® (GMAT ® ) test takers: American English vs. non-American English writers, English native speakers vs. English-as-a- second-language speakers, males vs. females, and examinees of three different ethnic groups. Propensity score matching was used to create control groups by matching each member of the studied groups on… 
Individual Fairness Evaluation for Automated Essay Scoring System
TLDR
This work proposes a methodology to measure individual fairness in Automated Essay Scoring (AES), and suggests that the Sentence-BERT, as the text representation of the essays, and Gradient Boosting, as a score prediction model, provide better results based on the proposed individual fairness evaluation methodology.
Global Times Call for Global Measures: Investigating Automated Essay Scoring in Linguistically-Diverse MOOCs.
TLDR
The findings from a linguistically-diverse pharmacy MOOC are reported on, which utilized an automated essay scoring (AES) assignment to engage students in the application of course content and suggested that the use of an AES system may disadvantage non-native English speakers.

References

SHOWING 1-8 OF 8 REFERENCES
An Evaluation of IntelliMetric™ Essay Scoring System
This report provides a two-part evaluation of the IntelliMetric™ automated essay scoring system based on its performance scoring essays from the Analytic Writing Assessment of the Graduate Management
Consider Propensity Scores to Compare Treatments
The underlying question when comparing treatments is usually whether an individual would do better with treatment X than they would with treatment Y. But there are often practical and theoretical
The central role of the propensity score in observational studies for causal effects
Abstract : The results of observational studies are often disputed because of nonrandom treatment assignment. For example, patients at greater risk may be overrepresented in some treatment group.
Constructing a Control Group Using Multivariate Matched Sampling Methods That Incorporate the Propensity Score
Abstract Matched sampling is a method for selecting units from a large reservoir of potential controls to produce a control group of modest size that is similar to a treated group with respect to the
G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences
TLDR
G*Power 3 provides improved effect size calculators and graphic options, supports both distribution-based and design-based input modes, and offers all types of power analyses in which users might be interested.
Matching using estimated propensity scores: relating theory to practice.
TLDR
These results delineate the wide range of settings in which matching on estimated linear propensity scores performs well, thereby providing useful information for the design of matching studies and applying theoretical approximations to practice.
An Overview of Automated Scoring of Essays.
TLDR
The main purpose of this article is to provide an overview of current approaches to AES and main characteristics of these systems will be discussed and current issues regarding the use of them both in low-stakes Assessment (in classrooms) and high-stakes assessment (as standardized tests) are discussed.
Statistical Power Analysis for the Behavioral Sciences (2nd ed.)