Loading Projects

« All Projects

Virtual Project Virtual Project
  • This project has passed.

S4: Advice and Information on Computational Reproductions

YSI Webinar on Replication

Start time:

October 13, 2022 @ 4:00 pm - 5:30 pm

Virtual Project Virtual Project
project Series Event Series (See All)

EDT

Location:

Online

Type:

Other

project Series Event Series (See All)
Virtual Project Virtual Project

Speakers

Speaker Image
Nate Breznau

Postdoctoral Fellow at the University of Bremen (DE)

Description

WEBINAR REPLICATE REPLICATE: HOW YOU CAN TAKE PART
SESSION 4: Advice and Information on Computational Reproductions: Students May Find Them Messier Than Expected

Drawing on experience teaching and conducting replications this talk focuses on the most basic form of replication: the computational reproduction. This means simply trying to reproduce the numerical results of a previous study using the same data and statistical routines from that study. As it turns out, there are many potential pitfalls that come from both the original study and the replicator, that could lead to different results. Different computing environments or data versioning and formatting are also culprits.

This talk reviews these pitfalls using examples from published literature and several attempts across disciplines to computationally reproduce findings published in journals. It also reports results from a crowdsourced replication, where eighty-five independent teams attempted a computational replication of results reported in an original study that is central to economics and many other disciplines on the links between policy preferences and immigration. Although when teams had the original data and code they were able to achieve a high rate of similar results (95.7%), a random half of teams did not get the original code and struggled (89.3%). What was more surprising was that exact numerical reproductions to the second decimal place were far less common (76.9% and 48.1%). This has wider implications for science obviously, but it should serve as a lesson for students using replication that things are not a clean cut as one might hope. (full paper on the crowdsourced study).
Please watch the pre-recorded video for this session here. The live session will then focus on discussion.
For discussion please use the session's page on the ReplicationWiki.

SHORT BIO
Nate Breznau, Postdoctoral Fellow at the University of Bremen. Researcher at the Comparative Research Center “The Global Dynamics of Social Policy”. Principal Investigator of the research project, “The Reciprocal Relationship of Public Opinion and Social Policy”. Principal Investigator of “The Crowdsourced Replication Initiative”. Open Science advocate. Open Science Fellow at Wikimedia with the project, “Giving the Results back to the Crowd”. User of preprints. Crowdsourcing and Mertonian-norm advocate. Author of the open science and crowdsourcing blog “Crowdid”.

Hosted by Working Group(s):

Attendees

Diana Soeiro

Jan H. Höffler

Ebele Nwokoye

Sabin NENDOBE DOBAH

Dina Moawad

Amelia Zajaczkowska

Geremew Milkias Waza

Oscar Jaramillo

Sangram Mane

R U Megha

Gustavo Castillo

Manuel Alfredo Barrientos Cifuentes

Franziska Strunz

Nate Breznau

Daniel Gotthardt

Thobile Mawelela