• Corpus ID: 17751958

Current Issues in the Determination of Usability Test Sample Size : How Many Users is Enough ?

@inproceedings{Turner2002CurrentII,
  title={Current Issues in the Determination of Usability Test Sample Size : How Many Users is Enough ?},
  author={Carl W. Turner and Jakob Nielsen},
  year={2002}
}
The topic of “how many users” is of great interest to usability specialists who need to balance project concerns over ROI and timelines with their own concerns about designing usable interfaces. The panel will review the current controversies over usability test sample size, test validity, and reliability. Introduction Virzi (1992), Nielsen and Landauer (1993), and Lewis (1994) published influential articles on the topic of sample size in usability testing. In these articles, the authors… 
Number of Subjects in Web Usability Testing
TLDR
This study provides evidence of the usefulness of SPRT in situations where determination of effectiveness is the goal, at a substantial reduction of number of users, and offers an alternative approach employing the Sequential Probability Ratio Test (SPRT) to determine product effectiveness.
Is a Website or E-Learning Product Working Well ? How Many Users Should You Test ?
How can one determine efficiently if an informational website or an e-learning product is working well? Relatively small numbers of the target audience are needed to improve a product during
Improved usability of electronic government services for the ageing population
TLDR
The research has demonstrated that improvements in the interface make e-government application possible which are perceived useful and easy to use by elderly users and developed a generalised solution, the IGUAN guideline.
A Participative Approach for Objective and Subjective Evaluation of Seat Discomfort and User Interface Usability
The main aim of the work is the objective and subjective evaluation of two aspects of the interaction user-product, the seating discomfort and the user interface usability, relevant to industrial
Expanding Interaction Potentials within Virtual Environments: Investigating the Usability of Speech and Manual Input Modes for Decoupled Interaction
TLDR
An evaluation of speech input across different user groups and a usability assessment of independent-user and collaborative-user interactions was conducted, demonstrating the potential usability issues of both input devices as well as highlighting their suitability for advanced virtual applications.
La calidad de los objetos de aprendizaje. Una aplicación de la plantilla GEHOA
Resumen. El presente trabajo describe el proceso de evaluación, por parte de expertos desde el punto de vista tecnológico y pedagógico, de un Objeto de Aprendizaje (OA) para un curso de Cálculo

References

SHOWING 1-10 OF 33 REFERENCES
Sample Sizes for Usability Studies: Additional Considerations
TLDR
Recently, Virzi (1992) presented data that support three claims regarding sample sizes for usability studies, providing evidence that the binomial probability formula may provide a good model for predicting problem discovery curves, given an estimate of the average likelihood of problem detection.
Relaxing the homogeneity assumption in usability testing
TLDR
Their geographical diversity is an excellent geographical spread but I cannot help but notice that the authors do not seem to attract authors from Africa or from South America, so I would like to take this opportunity to encourage any potential authors from these areas to submit papers.
Refining the Test Phase of Usability Evaluation: How Many Subjects Is Enough?
TLDR
Three experiments are reported in this paper that relate the proportion of usability problems identified in an evaluation to the number of subjects participating in that study, finding that 80% of the usability problems are detected with four or five subjects.
Measuring usability: are effectiveness, efficiency, and satisfaction really correlated?
TLDR
This work presents data from an experiment where 87 subjects solved 20 information retrieval tasks concerning programming problems, and the correlation between efficiency, as indicated by task completion time, and effectiveness, as indicating by quality of solution was negligible.
Introduction: Current Issues in Usability Evaluation
TLDR
In this introduction to the special issue of the International Journal of Human-Computer Interaction, some current topics in usability evaluation are discussed and how the contributions to the issue relate to these topics are indicated.
The Evaluator Effect: A Chilling Fact About Usability Evaluation Methods
TLDR
It is certainly notable that a substantial evaluator effect persists for evaluators who apply the strict procedure of CW or observe users thinking out loud, and it is highly questionable to use a TA with 1evaluator as an authoritative statement about what problems an interface contains.
A Cost-Effective Evaluation Method for Use by Designers
A mathematical model of the finding of usability problems
For 11 studies, we find that the detection of usability problems as a function of number of users tested or heuristic evaluators employed is well modeled as a Poisson process. The model can be used
Expert Reviews: How Many Experts is Enough?
TLDR
Five usability specialists were asked to review the user interface to a phone-based, interactive voice response system and showed that when allocating expert time, it is more effective to have a greater number of experts spend fewer hours than to use fewer experts for more hours.
Finding usability problems through heuristic evaluation
Usability specialists were better than non-specialists at performing heuristic evaluation, and “double experts” with specific expertise in the kind of interface being evaluated performed even better.
...
...