The MTurkification of Social and Personality Psychology

  title={The MTurkification of Social and Personality Psychology},
  author={Craig A. Anderson and Johnie J. Allen and Courtney N. Plante and Adele Quigley-McBride and Alison Lovett and J. Rokkum},
  journal={Personality and Social Psychology Bulletin},
  pages={842 - 850}
The potential role of brief online studies in changing the types of research and theories likely to evolve is examined in the context of earlier changes in theory and methods in social and personality psychology, changes that favored low-difficulty, high-volume studies. An evolutionary metaphor suggests that the current publication environment of social and personality psychology is a highly competitive one, and that academic survival and reproduction processes (getting a job, tenure/promotion… 

Figures and Tables from this paper

Research on climate change in social psychology publications: A systematic review

There is a strong scientific consensus that anthropogenic climate change is happening and that its impacts can put both ecological and human systems in jeopardy. Social psychology, the scientific study

Unburdening the Shoulders of Giants: A Quest for Disconnected Academic Psychology

  • Dario Krpan
  • Psychology
    Perspectives on psychological science : a journal of the Association for Psychological Science
  • 2020
It is argued that combining the two streams of psychology would result in the most significant advancement of psychological knowledge, and it is proposed that, alongside the currently existing connected psychology, disconnected psychology should be established.

Evaluating Research in Personality and Social Psychology: Considerations of Statistical Power and Concerns About False Findings

Data on the use of FFR calculations to update beliefs about the null or alternative hypothesis and assess the relative support for the null versus alternative hypothesis when evaluating a set of studies shows limits for statistical power in inflating false findings.

Perceptions and experiences of (people with) unconventional identities

ABSTRACT Psychology has long focused on social identities and their critical role in defining the self. However, the majority of identity-related findings stems from research on traditional

Extracting Agency and Communion From the Big Five: A Four-Way Competition

This research developed novel agency and communion scales based on the items from the most popular nonpropriety measure of the Big Five-the Big Five Inventory, and compared four alternative scale-construction methods: expert rating, target scale, ant colony, and brute force.

Improving Research on the Psychology of Sustainable Consumption: Some Considerations from an Early Career Perspective

Psychological research on sustainable consumption is developing a rich and diverse corpus of knowledge and tools, involving a broad range of disciplines. This very growth and diversity, however,

A Validity-Based Framework for Understanding Replication in Psychology

  • L. FabrigarD. WegenerR. Petty
  • Psychology, Business
    Personality and social psychology review : an official journal of the Society for Personality and Social Psychology, Inc
  • 2020
This article proposes that many aspects of this pattern of results can be understood within the classic framework of four proposed forms of validity: statistical conclusion validity, internal validity, construct validity, and external validity.

Research in Social Psychology Changed Between 2011 and 2016: Larger Sample Sizes, More Self-Report Measures, and More Online Studies

The debate about false positives in psychological research has led to a demand for higher statistical power. To meet this demand, researchers need to collect data from larger samples—which is

A sociocultural norm perspective on Big Five prediction.

The sociocultural norm perspective (SNP) provides norm-based, culture-focused, and mechanism-attentive explanations for why the Big Five predict their outcomes and dispels the view that such variation threatens the validity of the Big five.



Publication Trends in JPSP: A Three-Decade Review

To examine recent changes in research and publication practices, the authors systematically compared articles published in the Journal of Personality and Social Psychology during 1 968, 1978, and

Data collection in a flat world: the strengths and weaknesses of mechanical turk samples

MTurk offers a highly valuable opportunity for data collection, and it is recommended that researchers using MTurk include screening questions that gauge attention and language comprehension, avoid questions with factual answers, and consider how individual differences in financial and social domains may influence results.

Fifty Days an MTurk Worker: The Social and Motivational Context for Amazon Mechanical Turk Workers

  • G. B. Schmidt
  • Sociology
    Industrial and Organizational Psychology
  • 2015
The focal article of Landers and Behrend (2015) persuasively argues that universally condemning potential convenience data sources outside of traditional industrial–organizational (I-O) samples such

A Convenient Solution: Using MTurk To Sample From Hard-To-Reach Populations

We agree with Landers and Behrend's (2015) proposition that Amazon's Mechanical Turk (MTurk) may provide great opportunities for organizational research samples. However, some groups are

Caution! MTurk Workers Ahead—Fines Doubled

There is only scant evidence demonstrating that online sources can provide usable data for organizational research and that there is a need for further research evaluating the validity of these new sources of data.

Amazon Mechanical Turk in Organizational Psychology: An Evaluation and Practical Recommendations

PurposeAmazon Mechanical Turk is an increasingly popular data source in the organizational psychology research community. This paper presents an evaluation of MTurk and provides a set of practical

Best research practices in psychology: Illustrating epistemological and pragmatic considerations with the case of relationship science.

This article contrasts the false-positives-reduction (FPR) approach with an alternative, the error balance (EB) approach, which argues that any serious consideration of optimal scientific practice must contend simultaneously with both false-positive and false-negative errors.

Attentive Turkers: MTurk participants perform better on online attention checks than do subject pool participants

In three online studies, participants from MTurk and collegiate populations participated in a task that included a measure of attentiveness to instructions (an instructional manipulation check: IMC), and MTurkers were more attentive to the instructions than were college students, even on novel IMCs.

An Inconvenient Truth: Arbitrary Distinctions Between Organizational, Mechanical Turk, and Other Convenience Samples

Sampling strategy has critical implications for the validity of a researcher's conclusions. Despite this, sampling is frequently neglected in research methods textbooks, during the research design