Investigating variation in replicability: A "many labs" replication project


Richard A. Klein, University of Florida
Kate A. Ratliff, University of Florida
Michelangelo Vianello, Università degli Studi di Padova
Reginald B. Adams, Pennsylvania State University
Štěpán Bahník, Julius-Maximilians-Universität Würzburg
Michael J. Bernstein, Penn State Abington
Konrad Bocian, SWPS Uniwersytet Humanistycznospołeczny
Mark J. Brandt, Tilburg University
Beach Brooks, University of Florida
Claudia Chloe Brumbaugh, City University of New York
Zeynep Cemalcilar, Koç Üniversitesi
Jesse Chandler, University of Michigan, Ann Arbor
Winnee Cheong, HELP University
William E. Davis, Texas A&M University
Thierry Devos, San Diego State University
Matthew Eisner, University of Michigan, Ann Arbor
Natalia Frankowska, SWPS Uniwersytet Humanistycznospołeczny
David Furrow, Mount Saint Vincent University
Elisa Maria Galliani, Università degli Studi di Padova
Fred Hasselman, Radboud University Nijmegen
Joshua A. Hicks, Texas A&M University
James F. Hovermale, Virginia Commonwealth University
S. Jane Hunt, Texas A&M University-Commerce
Jeffrey R. Huntsinger, Loyola University of Chicago
Hans Ijzerman, Tilburg University
Melissa Sue John, Worcester Polytechnic Institute
Jennifer A. Joy-Gaba, Virginia Commonwealth University
Heather Barry Kappes, London School of Economics and Political Science
Lacy E. Krueger, Texas A&M University-Commerce
Jaime Kurtz, James Madison University
Carmel A. Levitan, Occidental College
Robyn K. Mallett, Loyola University of Chicago
Wendy L. Morris, McDaniel College

Document Type


Publication Date



Although replication is a central tenet of science, direct replications are rare in psychology. This research tested variation in the replicability of 13 classic and contemporary effects across 36 independent samples totaling 6,344 participants. In the aggregate, 10 effects replicated consistently. One effect - imagined contact reducing prejudice - showed weak support for replicability. And two effects - flag priming influencing conservatism and currency priming influencing system justification - did not replicate. We compared whether the conditions such as lab versus online or US versus international sample predicted effect magnitudes. By and large they did not. The results of this small sample of effects suggest that replicability is more dependent on the effect itself than on the sample and setting used to investigate the effect. © 2014.

Publication Name

Social Psychology

Volume Number


First Page


Last Page


Issue Number




This document is currently not available here.