Testing a hypothesis can typically be done with many different kinds of study designs. Choosing a particular design constitutes a source of uncertainty in estimating the underlying effect size which is not incorporated into common research practices. In a new paper just published in PNAS led by SSE Department of Economics Professors Anna Dreber and Magnus Johannesson together with Christoph Huber at WU Vienna University of Economics and Business, Utz Weitzel at Vrije University Amsterdam, and Felix Holzmeister, Juergen and Michael Kirchler at University of Innsbruck, the focus is on exploring this uncertainty in a crowd-sourced project. 45 independent research teams proposed research designs to address the same research question in monetarily incentivized experiments: Does competition affect moral behavior? More than 18,000 participants were randomized to different designs and the results were studied in a pre-registered meta-analysis. The results suggest a small negative effect of competition on moral behavior. Importantly, however, the variation in effect size estimates across the 45 designs is substantially larger than the variation expected due to sampling errors. This “design heterogeneity” highlights that the generalizability and informativeness of individual experimental designs is limited.