The pitfall of experimenting on the web: How unattended selective attrition leads to surprising (yet false) research conclusions.

@article{Zhou2016ThePO,
  title={The pitfall of experimenting on the web: How unattended selective attrition leads to surprising (yet false) research conclusions.},
  author={Haotian Zhou and A. Fishbach},
  journal={Journal of personality and social psychology},
  year={2016},
  volume={111 4},
  pages={
          493-504
        }
}
The authors find that experimental studies using online samples (e.g., MTurk) often violate the assumption of random assignment, because participant attrition-quitting a study before completing it and getting paid-is not only prevalent, but also varies systemically across experimental conditions. Using standard social psychology paradigms (e.g., ego-depletion, construal level), they observed attrition rates ranging from 30% to 50% (Study 1). The authors show that failing to attend to attrition… Expand
216 Citations
Show Me the Money: A Systematic Exploration of Manipulations, Moderators, and Mechanisms of Priming Effects
Validity and Mechanical Turk: An assessment of exclusion methods and interactive experiments
Moving Beyond Unwise Replication Practices: The Case of Romantic Motivation
Attrition from Web-Based Cognitive Testing: A Repeated Measures Comparison of Gamification Techniques
Lie for a Dime
Illusory-Correlation Effects on Implicit and Explicit Evaluation.
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 24 REFERENCES
Data Collection in a Flat World: The Strengths and Weaknesses of Mechanical Turk Samples
The online laboratory: conducting experiments in a real labor market
Regulatory Focus in Predictions About Others
Amazon's Mechanical Turk
Mediated Models for the Analysis of Confounded Variables and Self-Selected Samples
...
1
2
3
...