Automatic Assessment of Language Proficiency through Shadowing

  title={Automatic Assessment of Language Proficiency through Shadowing},
  author={Dean Luo and Nobuaki Minematsu and Yutaka Yamauchi and Keikichi Hirose},
  journal={2008 6th International Symposium on Chinese Spoken Language Processing},
  • Dean Luo, N. Minematsu, K. Hirose
  • Published 30 December 2008
  • Linguistics
  • 2008 6th International Symposium on Chinese Spoken Language Processing
Shadowing is a practice that requires learners to shadow a presented native utterance as closely and quickly as possible. Learners' pronunciation in shadowing, especially in the case of beginners, often becomes inarticulate and corrupt. These features of shadowing make it very difficult to assess shadowing productions. In this paper, we investigate the automatic pronunciation scoring methods for shadowing. Three automatic scores have be proposed and compared with each other. Experiments show… 

Figures from this paper

DNN-Based Scoring of Language Learners’ Proficiency Using Learners’ Shadowings and Native Listeners’ Responsive Shadowings
Experiments show that the correlation between the DNN-based predicted scores and the averaged human scores is higher than or at least comparable to the averaged correlation betweenThe scores of human raters, indicating that the proposed automatic rating module can be introduced to language education as another human rater.
Automatic Assessment and Error Detection of Shadowing Speech: Case of English Spoken by Japanese Learners
This study attempts to improve automatic assessment of shadowing speech and give corrective feedbacks to learners based on error detection, and uses GOP scores, Word Recognition Rate, silence ratio, forced alignment log-likelihood scores, and word omission rate to predict the overall proficiency of the individual speakers.
A Corpus-based Analysis of Shadowing Speech: Case of L2 English by Japanese Learners
In this study we intend to investigate the typical phenomena in shadowing speech and work out a tentative scheme for shadowing speech labeling. Our aim is two-fold: a) to give useful feedback to
A Study of Objective Measurement of Comprehensibility through Native Speakers' Shadowing of Learners' Utterances
Results suggest it is valid to regard comprehensible pronunciation as shadowable pronunciation and shadowers’ subjective assessment of learners’ speech and that of their shadowings are highly correlated and that the former is more correlated with the GOP scores of natives’ shadowings than those of learners' speech.
Automatic Scoring of Shadowing Speech Based on DNN Posteriors and Their DTW
Experiments suggest that, compared to HMM, DNN can improve teacher-machine correlation largely by 0.37 and DTW based on DNN posteriors shows as high correlation as 0.74 even when posterior calculation is done using a different language from the target language of learning.
DNN-based GOP and Its Application to Automatic Assessment of Shadowing Speeches
This study collected English shadowing speeches from Japanese university students and investigated automatic proficiency assessment using DNN (Deep Neural Network) based acoustic models, suggesting that DNN based approach shows better performance than traditional GMM based ones.
Investigation of teacher-selected sentences and machine-suggested sentences in terms of correlation between human ratings and GOP-based machine scores
This study investigated relationships between teacher-selected stimulus sentences and machine-suggested ones in terms of the correlation between human ratings and GOP-based machine scores and found that some particular sentences frequently appeared in the sets and seemed to have something to do with maximizing correlation.
Predicting Second Language Proficiency Level Using Linguistic Cognitive Task and Machine Learning Techniques
A novel method for predicting second language proficiency based on linguistic cognitive ability measured in linguistic cognitive response test based on an assumption that there is a correlation between language aptitude test scores and linguistic Cognitive ability is proposed.
Unsupervised optimal phoneme segmentation: theory and experimental evaluation
The authors formulate the unsupervised segmentation as an optimal problem by means of maximum likelihood, and show that the optimal segmentation corresponds to minimising the coding length of the input sequence.


Sensei: Spoken language assessment for call center agents
This paper describes how the individual parameters of spoken English skills are assessed in Sensei using various speech processing techniques and the experiments conducted to evaluate these techniques and how the performance is compared with assessment performed by human assessors.
Although speech derived from reading texts, and similar types of speech, e.g. that from reading newspapers or that from news broadcast, can be recognized with high accuracy, recognition accuracy
Unsupervised optimal phoneme segmentation: Objectives, algorithm and comparisons
This paper introduces a time-constrained agglomerative clustering algorithm to find the optimal segmentations and proposes an efficient method to implement the algorithm by using integration functions.