The Texas sharpshooter fallacy.

Abstract

The ‘Texas sharpshooter fallacy’ (or: ‘clustering illusion’) refers to a man with a gun but no shooting skills who fires a number of bullets into the wall of a barn, paints a bull’s-eye around a fortuitous cluster of bullet holes, and declares himself a sharpshooter (Smith, 2016). In every collection of data there will occur coincidental clusters, just as in one hundred random gunshots there will be clusters of bullet holes. Human perception tends to identify patterns where none actually exist. We ignore differences and focus on similarities, thus increasing the risk of arriving at false conclusions. For scientific findings to be convincing, the target should be pre-specified (the bull’s-eye should be painted before firing the bullets): Does endometriosis occur more often in red-haired Caucasian women between 26 and 33 years of age? If a large enough prospective study confirms this pre-specified detailed study question, you are in business. If the red-haired, middle-aged Caucasian emerges as the result of intensive data dredging and torturing of immense administrative databases for nominally significant associations, comparing dozens of attributes without correcting for multiple observations, you are in trouble. Especially, if a pathophysiological rationale for your research is lacking and a biological plausibility of your findings does not exist (Braakhekke et al., 2017). Forecasting endometriosis after having performed a laparoscopy is easy. Predictive factors (e.g. biomarkers) may be developed in one set of patients; they should be validated in a second, independent set of similar patients. Prognostic models always do well in the population they were developed in. When the same set of data is used to create and to test a hypothesis, the Texas sharpshooter chuckles. The majority of research is redundant (Macleod et al., 2014). It has been estimated that 85% of research funding is wasted, by inappropriate research questions, irrelevant endpoints, faulty study design and flawed execution, poor reporting and non-publication (Moher et al., 2016). We should focus on increasing value and reducing waste (Glasziou et al., 2014; Ioannidis et al., 2014). If this means that ‘Human Reproduction’ will be slimmer in the future, so be it. We want to offer top quality research that clinicians and patients can trust and rely on. To quote Douglas Altman: ‘What is needed is less research, better research and research performed for the right reasons’.

DOI: 10.1093/humrep/dex103

Cite this paper

@article{Evers2017TheTS, title={The Texas sharpshooter fallacy.}, author={Johannes Evers}, journal={Human reproduction}, year={2017}, volume={32 7}, pages={1363} }