Presentation Title

The Many Labs 5 (ML5) Replication Study: A Possible Method for Improving Replication Rates in Psychology

Faculty Mentor

Máire Ford

Start Date

18-11-2017 12:30 PM

End Date

18-11-2017 1:30 PM

Location

BSC-Ursa Minor 19

Session

Poster 2

Type of Presentation

Poster

Subject Area

behavioral_social_sciences

Abstract

This project is a replication study, that is part of a larger project known as the Many Labs 5 Replication Study (ML5), being conducted across many labs internationally. The ML5 study is a follow-up to another large project called The Reproducibility Project: Psychology (RPP), which investigated how well published studies in psychological science replicate. Researchers attempted to replicate 100 psychological studies, but only about 40% replicated. Eleven of the studies that did not replicate were labeled as “non-endorsed” (meaning prior to running the replication the original author expressed concerns about the methodology). These findings led to the development of the ML5 project. This project reexamines the reproducibility of the 11 “non-endorsed” replications. Researchers are especially interested in whether peer review of research protocols prior to running a replication study will improve the quality of the methodology, and subsequently improve replication rates. To do this, all protocols were peer reviewed prior to data collection to ensure they adhered to the original study. This component of the study is particularly important because it isOf the 11, the research group at LMU, headed by Dr. Maire Ford, was assigned the replication of a study examining the ability of 3-year-old children and adults to identify threatening animal (snakes) versus nonthreatening animal (frogs; LoBue & DeLoache, 2008). Participants completed a computer reaction time task where their goal was to identify either a threatening animal or a non-threatening animal. Our data was sent to the lead researcher who will analyze data across several replication sites to determine if the findings replicate. If findings do replicate, this means peer review of the protocol prior to conducting the study improves replicability. This might further suggest the low replication rate in the RPP was due to poor study design, rather than irreplaceable findings. Since the data has not been analyzed yet, there are no results to report. Thus, I will be presenting an overview of the replication project and its importance.

Summary of research results to be presented

The role of Dr. Máire Ford's lab in this larger project was primarily data collection. Since there were three other labs assigned to do the same replication, a researcher from one of the other three labs was designated as the lead researcher. Once data collection was finished, the data was sent to the assigned lead researcher. This individual will analyze data across the replication sites to determine if the findings replicate. Although data analysis is still in the process of being conducted, this research and its subsequent results will have significant contributions for the ongoing discussions regarding the replication crisis within the field of psychology.

This document is currently not available here.

Share

COinS
 
Nov 18th, 12:30 PM Nov 18th, 1:30 PM

The Many Labs 5 (ML5) Replication Study: A Possible Method for Improving Replication Rates in Psychology

BSC-Ursa Minor 19

This project is a replication study, that is part of a larger project known as the Many Labs 5 Replication Study (ML5), being conducted across many labs internationally. The ML5 study is a follow-up to another large project called The Reproducibility Project: Psychology (RPP), which investigated how well published studies in psychological science replicate. Researchers attempted to replicate 100 psychological studies, but only about 40% replicated. Eleven of the studies that did not replicate were labeled as “non-endorsed” (meaning prior to running the replication the original author expressed concerns about the methodology). These findings led to the development of the ML5 project. This project reexamines the reproducibility of the 11 “non-endorsed” replications. Researchers are especially interested in whether peer review of research protocols prior to running a replication study will improve the quality of the methodology, and subsequently improve replication rates. To do this, all protocols were peer reviewed prior to data collection to ensure they adhered to the original study. This component of the study is particularly important because it isOf the 11, the research group at LMU, headed by Dr. Maire Ford, was assigned the replication of a study examining the ability of 3-year-old children and adults to identify threatening animal (snakes) versus nonthreatening animal (frogs; LoBue & DeLoache, 2008). Participants completed a computer reaction time task where their goal was to identify either a threatening animal or a non-threatening animal. Our data was sent to the lead researcher who will analyze data across several replication sites to determine if the findings replicate. If findings do replicate, this means peer review of the protocol prior to conducting the study improves replicability. This might further suggest the low replication rate in the RPP was due to poor study design, rather than irreplaceable findings. Since the data has not been analyzed yet, there are no results to report. Thus, I will be presenting an overview of the replication project and its importance.