Quantifying Qualitative Analyses of Verbal Data: A Practical Guide

Abstract

This article provides one example of a method of analyzing qualitative data in an objective and quantifiable way. Although the application of the method is illustrated in the context of verbal data such as explanations, interviews, problem-solving protocols, and retrospective reports, in principle, the mechanics of the method can be adapted for coding other types of qualitative data such as gestures and videotapes. The mechanics of the method we outlined in 8 concrete step. Although verbal analyses can be used for many purposes, the main goal of the analyses discussed here is to formulate an understanding of the representation of the knowledge used in cognitive performances and how that representation changes with learning This can be contrasted with another method or analyzing verbal protocols, the goal of which is to validate the cognitive processes of human performance, often as embodied in a computational model. For a variety of reasons, there has been an increasing need in cognitive science and educational research to collect and analyze "messy" data. Messy data refer to such things as verbal explanations, observations, and videotapings, as well as gestures. One reason for the need to collect this kind of data is the trend toward studying complex activities in practice or in the context in which they occur. So, for example, to understand how an apprentice learns a trade, one might have to observe the learner in context. Likewise, it is becoming increasingly clear that the performance of experts (such as industrial and software designers) relies on the use of external aids and tools, such as notes and drawings (Norman, 1988). Thus, to capture a complete understanding of their skill, ideally one should incorporate in the analysis not only their verbal transcripts but also their drawings, pointings, and gesturings (Tang, 1989). Of course, both verbal data and observational data have been used widely for some time, in cognitive simulation research for the former case and in anthropological studies for the latter. However, it has been discouraging for novice students of cognitive science and education to adopt these methods for various reasons, such as the restricted applicability of the protocol analysis method (see Ericsson & Simon, 1984), the subjectiveness of the observational methods (see Schofield & Anderson, 1987), and the time-consumingness of both of these methods. The goal of this article is to attempt to provide guidance for how one can approach an analysis of verbal data more …

3 Figures and Tables

Showing 1-10 of 53 references

The content of physics self-explanations

  • M T H Chi, K A Vanlehn
  • 1991
Highly Influential
4 Excerpts

Individual differences in the solving of social science problems

  • J F Voss, S W Tyler, L A Yengo
  • 1983
Highly Influential
4 Excerpts

Network representation of a child's dinosaur knowledge

  • M T H Chi, R Koeske
  • 1983
Highly Influential
5 Excerpts

Protocol analysis: Verbal reports as data (Rev

  • K A Ericsson, H Simon
  • 1993
Highly Influential
4 Excerpts

Stolen knowledge: What is teamed in the context of a medical intensive care unit

  • M T H Chi, A Hashem, S Ludvigsen, V Shalin, D Bertram
  • 1997

Constructing self-explanations and scaffolded explanations in tutoring

  • M T H Chi
  • 1996
2 Excerpts

Understanding constraint-based processes: A precursor to conceptual change in physics

  • J D Slotta, M T H Chi
  • 1996
1 Excerpt

Assessing students' misclassifications of physics concepts: The ontological basis of conceptual change

  • J D Slotta, M T H Chi, E Joram
  • 1995
1 Excerpt
Showing 1-10 of 224 extracted citations
0204060'96'98'00'02'04'06'08'10'12'14'16
Citations per Year

492 Citations

Semantic Scholar estimates that this publication has received between 378 and 637 citations based on the available data.

See our FAQ for additional information.