Reproducibility and responsiveness of evaluative outcome measures. Theoretical considerations illustrated by an empirical example.
A critique of Hankins, M: 'How discriminating are discriminative instruments?' Health and Quality of Life Outcomes 2008, 6:36. Background Recently Hankins (re-)introduced Ferguson's coefficient δ as an index of discrimination, to be distinguished from the well-known measurement properties validity and reliability [1,2]. Hankins presented Ferguson's δ as a useful index of the degree to which an instrument discriminates between individuals, being "the ratio of the observed number of between-person differences to the theoretical maximum number possible" . The value of δ varies between 0 (no discrimination at all) and 1 (maximal possible discrimination). The calculation is straightforward and Hankins provided a generalized formula for calculating δ for questionnaires with dichotomous as well as polytomous items. Hankins' paper  elicited two critical comments [3,4]. Wyrwich referred to the work of Guyatt  who related discrimination tot reliability, theoretically consistent correlations with other measures, and interpretability of small but important differences. Since Hankins failed to present relevant information regarding these issues, Wyrwich concluded that it is impossible to make a judgement on whether Ferguson's δ is a useful index or not . Whereas Hankins stated that discrimination is something else than reliability, Norman expressed the opposite view, i.e. that "reliability is discrimination". Scrutinizing Hankins' examples and adding one of his own, Norman illustrated his main point that Ferguson's δ fails to distinguish between true differences and measurement error . In his response, Hankins remarked that both Norman and Wyrwich made too much of his examples, and seemed to have missed his point, which is that Ferguson's δ is an additional index of an instruments' measurement properties, beside reliability, validity and interpretability, and that Ferguson's δ can only be computed on the assumption that the measurement is valid and reliable . In this letter, we will examine how exactly Ferguson's δ 'works' and what δ actually measures. More specifically, we will show that the magnitude of δ is only determined by the distribution of the scores in a given sample. Moreover, we will show that the standard computation of δ ignores reliability, but, when reliability is accounted for, δ becomes impossible to interpret. Our final conclusion will be that Ferguson's δ is not a useful attribute of a measurement instrument. How Ferguson's δ works The formula of δ, presented by Hankins, reads: Published: 30 April 2009 Health and Quality of Life Outcomes 2009, 7:38 doi:10.1186/1477-7525-7-38 Received: 23 February 2009 Accepted: 30 April 2009 This article is available from: http://www.hqlo.com/content/7/1/38 © 2009 Terluin et al; licensee BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.