Promises and Perils of Experimentation: The Mutual-Internal-Validity Problem

  title={Promises and Perils of Experimentation: The Mutual-Internal-Validity Problem},
  author={Hause Lin and Kaitlyn M. Werner and Michael Inzlicht},
  journal={Perspectives on Psychological Science},
  pages={854 - 863}
Researchers run experiments to test theories, search for and document phenomena, develop theories, or advise policymakers. When testing theories, experiments must be internally valid but do not have to be externally valid. However, when experiments are used to search for and document phenomena, develop theories, or advise policymakers, external validity matters. Conflating these goals and failing to recognize their tensions with validity concerns can lead to problems with theorizing… Expand
Beyond Statistical Ritual: Theory in Psychological Science
  • T. Proulx, R. Morey
  • Medicine
  • Perspectives on psychological science : a journal of the Association for Psychological Science
  • 2021
It is suggested that psychologists have too often chosen the latter option, substituting synthetic certainties for theory-guided research, in much the same manner as Scholastic scholars did centuries ago. Expand
Labor/leisure decisions in their natural context: The case of the smartphone
Using a situated approach to studying labor/leisure decisions, the findings highlight the importance of high task motivation, as well as the temporary distractive potential of smartphone notifications, when people face the challenge of staying focused on their productive tasks. Expand
Do early birds share their worms? How prosocial behaviour and empathy vary across the day
Abstract Social behaviours have been proposed to follow circadian patterns across the course of the day, but empirical support is mixed. As a within-subject test of circadian effects on everydayExpand
Effectiveness of a Self-Fitting Tool for User-Driven Fitting of Hearing Aids
A digital self-fitting tool enables multiple sessions and easy re-fitting, with the potential to outperform the classical fitting approach, and quasi-equivalence of the performance of the classical audiological fitting approach is reported. Expand


Artificiality: The tension between internal and external validity in economic experiments
The artificiality of a laboratory situation is placed in the context of the tension between external and internal validity. Most economists consider internal validity to be most important. A properExpand
Theory Before the Test: How to Build High-Verisimilitude Explanatory Theories in Psychological Science
It is suggested that psychological science, by focusing on effects, may lose sight of its primary explananda: psychological capacities, and how to leverage effects to understand capacities better is explained. Expand
The generalizability crisis.
  • T. Yarkoni
  • Medicine
  • The Behavioral and brain sciences
  • 2020
It is argued that failure to take the alignment between verbal and statistical expressions seriously lies at the heart of many of psychology's ongoing problems (e.g., the replication crisis), and concludes with a discussion of several potential avenues for improvement. Expand
What Do Laboratory Experiments Tell Us About the Real World
An important question facing experimental economists is whether behavior inside the laboratory is a good indicator of behavior outside the laboratory. We begin with a model that assumes the choicesExpand
Replicating patterns of prospect theory for decision under risk
In a sample of over 4,000 participants from 19 countries, the core patterns from a highly influential study on behaviour and decision-making broadly replicate, with only minor exceptions and somewhat smaller effect sizes. Expand
Experimental practices in economics: A methodological challenge for psychologists?
There is a call for more research on the consequences of methodological preferences, such as the use on monetary payments, and a “do-it-both-ways” rule regarding the enactment of scripts, repetition of trials, and performance-based monetary payments. Expand
Experiments in economics: External validity and the robustness of phenomena
External validity is the problem of generalizing results from laboratory to non‐laboratory conditions. In this paper we review various ways in which the problem can be tackled, depending on the kindExpand
Estimating the reproducibility of psychological science
A large-scale assessment suggests that experimental reproducibility in psychology leaves a lot to be desired, and correlational tests suggest that replication success was better predicted by the strength of original evidence than by characteristics of the original and replication teams. Expand
In defense of external invalidity.
Many psychological investigations are accused of "failure to generalize to the real world" because of sample bias or artificiality of setting. It is argued in this article that such "generalizations"Expand
Experimental economics and the artificiality of alteration
A neglected critique of social science laboratories alleges that they implement phenomena different to those supposedly under investigation. The critique purports to be conceptual and so invulnerableExpand