Two Case Studies in Using Cognitive Walkthrough for Interface Evaluation

@inproceedings{Jacobsen2000TwoCS,
  title={Two Case Studies in Using Cognitive Walkthrough for Interface Evaluation},
  author={Niels Ebbe Jacobsen and Bonnie E. John},
  year={2000}
}
Previous HCI-studies have compared usability evaluation methods quantitatively without supplementing these data with detailed qualitative data about how analysts actually learn and use methods. [...] Key Result Further, we recommend developing a tool to reduce the tedium and integrate our suggested improvements to CW.Expand
View via Publisher
reports-archive.adm.cs.cmu.edu
44 Citations
Highly Influential Citations
1
Background Citations
11
Methods Citations
11

44 Citations

Citation Type

Citation Type

Two psychology-based usability inspection techniques studied in a diary experiment
TLDR
An experiment comparing two psychology-based inspection techniques, cognitive walkthrough (CW) and metaphors of human thinking (MOT) is presented, showing that participants find it hard to understand MOT, while CW limits the scope of their search for usability problems. Expand
Scoping Analytical Usability Evaluation Methods: A Case Study
TLDR
A detailed comparison of eight analytical UEMs applied to a robotic arm interface and the findings were systematically compared against video data of the arm in use, yielding insights into the scope of each method. Expand
The Evaluator Effect: A Chilling Fact About Usability Evaluation Methods
TLDR
It is certainly notable that a substantial evaluator effect persists for evaluators who apply the strict procedure of CW or observe users thinking out loud, and it is highly questionable to use a TA with 1evaluator as an authoritative statement about what problems an interface contains. Expand
The Evaluator Effect: A Chilling Fact About Usability Evaluation Methods
TLDR
It is certainly notable that a substantial evaluator effect persists for evaluators who apply the strict procedure of CW or observe users thinking out loud, and it is highly questionable to use a TA with 1evaluator as an authoritative statement about what problems an interface contains. Expand
Adapting the cognitive walkthrough method to assess the usability of a knowledge domain visualization
TLDR
How the traditional CW method may be adapted for assessing the usability of knowledge domain visualization systems is described and the adapted version of CW is applied to CiteSpace, a KDViz tool that uses bibliometric analyses to create visualizations of scientific literatures. Expand
What do usability evaluators do in practice?: an explorative study of think-aloud testing
TLDR
The study shows how practical realities rarely discussed in the literature on usability evaluation influence sessions, and discusses implications for usability researchers and professionals, including techniques for fast-paced analysis and tools for capturing observations during sessions. Expand
Integration of Usability Evaluation Studies via a Novel Meta-Analytic Approach: What are Significant Attributes for Effective Evaluation?
The overall discovery rates, which are the ratios of sum of unique usability problems detected by all experiment participants against the number of usability problems existed in the evaluated… Expand
Dogmas in the assessment of usability evaluation methods
TLDR
Detailed studies of evaluation processes, assessments of the impact of UEMs on design carried out in real-world systems development and analyses of how U EMs may be combined are emphasised. Expand
The Collaborative Critique: An Inspection Method for Expert Evaluation of User Interfaces
TLDR
The CC method consists of a walkthrough evaluation in which the questions answered at each step of the interaction assess the system’s collaborative strength and its usefulness in evaluating an interface. Expand
Understanding Your Users: A Practical Guide to User Research Methods
This new and completely updated edition is a comprehensive, easy-to-read, "how-to" guide on user research methods. You'll learn about many distinct user research methods and also pre- and post-method… Expand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 57 REFERENCES
Evaluating a Multimedia Authoring Tool
TLDR
Several hypotheses about the cause of low effectiveness are presented, which suggest that additheory and software engineering practices: Wharton, Rietional knowledge, currently tacit in CW-experts, could be man. Expand
Applying cognitive walkthroughs to more complex user interfaces: experiences, issues, and recommendations
TLDR
It is believed that if the Cognitive Walkthrough is ultimately to be successful in industrial settings, the method must be refined and augmented in a variety of ways. Expand
Understanding Usability Issues Addressed by Three User-System Interface Evaluation Techniques
TLDR
Results showed that the cognitive walkthrough method identifies issues almost exclusively within the action specification stage, while guidelines covered more stages, and all the techniques could be improved in assessing semantic distance and addressing all stages on the evaluation side of the HCI activity cycle. Expand
Testing a walkthrough methodology for theory-based design of walk-up-and-use interfaces
TLDR
A “cognitive walkthrough” procedure for systematically evaluating features of an interface in the context of the theory and consideration of the walkthrough sheds light on the consistency with which such a procedure can be applied as well as the accuracy of the results. Expand
Tracking the effectiveness of usability evaluation methods
TLDR
A case study that tracks usability problems predicted with six usability evaluation methods through a development process and concludes that Predictive methods are not as effective as the HCI field would like. Expand
Comparison of empirical testing and walkthrough methods in user interface evaluation
TLDR
The findings were replicated across the two systems and show that the empirical testing condition identified the largest number of problems, and identified a significant number of relatively severe problems that were missed by the walkthrough conditions. Expand
The diary study: a workplace-oriented research tool to guide laboratory efforts
  • J. Rieman
  • Computer Science, Psychology
  • INTERCHI
  • 1993
TLDR
The diary study technique is described, a tool for research in the workplace that achieves a relatively high standard of objectivity and a diary study is reported that focuses on exploratory learning. Expand
Cognitive Walkthroughs: A Method for Theory-Based Evaluation of User Interfaces
TLDR
The cognitive walkthrough methodology, described in detail, is an adaptation of the design walkthrough techniques that have been used for many years in the software engineering community and is based on a theory of learning by exploration presented. Expand
The Evaluator Effect during First-Time Use of the Cognitive Walkthrough Technique
TLDR
While several studies have evaluated how well CW predicts the problems encountered in thinking-aloud studies, data from Lewis et al. suggests that the variability in performance among evaluators using CW is much lower than that of evaluator using heuristic evaluation or thinking- aloud studies (Jacobsen and Nielsen 1998, Nielsen 1994). Expand
Learning and using the cognitive walkthrough method: a case study approach
TLDR
This study results in several clear messages to both system designers and developers of evaluation techniques: this technique is currenlty learnable and usable, but there are several areas where further methoddevelopment would greatly contribute to a designer's use of the technique. Expand
...
1
2
3
4
5
...