<aside> <img src="/icons/redirect_blue.svg" alt="/icons/redirect_blue.svg" width="40px" />
Post-meeting Resources
📃Slides:
Preregistration Slides by Kim.pptx
🎦Zoom Recording
</aside>
Date: October 2025
Presenter: Kimberly Quinn
Learning Community: Open Science in Practice: Tools and Workflows for Transparent, Reproducible Research
By the end of this session, participants will:
<aside> <img src="/icons/info-alternate_blue.svg" alt="/icons/info-alternate_blue.svg" width="40px" />
[Recommended] Nosek, B. A., Beck, E. D., Campbell, L., Flake, J. K., Hardwicke, T. E., Mellor, D. T., van 't Veer, A. E., & Vazire, S. (2019). Preregistration is hard, and worthwhile. Trends in Cognitive Sciences, 23(10), 815-818. https://doi.org/10.1016/j.tics.2019.07.009
A concise article discussing the benefits and challenges of preregistration, emphasizing that it's a skill requiring practice and that deviations should be transparently reported.
van den Akker, O. R., van Assen, M. A. L. M., Bakker, M., Elsherif, M., Wong, T. K., & Wicherts, J. M. (2024). Preregistration in practice: A comparison of preregistered and non-preregistered studies in psychology. Behavior Research Methods, 56, 5424-5433. https://doi.org/10.3758/s13428-023-02277-0
Compares 193 preregistered psychology studies to 193 non-preregistered studies, finding that preregistered studies had larger sample sizes and more power analyses, though similar rates of positive results.
Willroth, E. C., & Atherton, O. E. (2024). Best laid plans: A guide to reporting preregistration deviations. Advances in Methods and Practices in Psychological Science, 7(1), 1-14. https://doi.org/10.1177/25152459231213802
Provides a comprehensive framework and template for transparently reporting deviations from preregistered plans, including survey results from 34 psychology journal editors.
Willroth_2024_Best laid plans - A guide to reporting preregistration deviations.pdf
Bakker, M., Veldkamp, C. L. S., van Assen, M. A. L. M., Crompvoets, E. A. V., Ong, H. H., Nosek, B. A., Soderberg, C. K., Mellor, D., & Wicherts, J. M. (2020). Ensuring the quality and specificity of preregistrations. PLOS Biology, 18(12), e3000937. https://doi.org/10.1371/journal.pbio.3000937
Compares two preregistration formats (structured vs. unstructured) to evaluate how effectively they restrict researcher degrees of freedom, finding that structured formats with detailed guidance better reduce opportunistic use of flexibility in research decisions.
Bakker_2020_Ensuring the quality and specificity of preregistrations.pdf
Frankenhuis, W. E., & Nettle, D. (2018). Open science is liberating and can foster creativity. Perspectives on Psychological Science, 13(4), 439-447. https://doi.org/10.1177/1745691618767878
Argues that Open Science practices, including preregistration, are liberating rather than constraining for researchers, as they enable transparent exploration, reward quality over outcomes, and can foster a more collaborative and creative research environment.
Frankenhuis_2018_PoPS_Open science is liberating and can foster creativity.pdf
McKiernan, E. C., Bourne, P. E., Brown, C. T., Buck, S., Kenall, A., Lin, J., McDougall, D., Nosek, B. A., Ram, K., Soderberg, C. K., Spies, J. R., Thaney, K., Updegrove, A., Woo, K. H., & Yarkoni, T. (2016). How open science helps researchers succeed. eLife, 5, e16800. https://doi.org/10.7554/eLife.16800
Reviews evidence demonstrating that open research practices, including open access publishing and data sharing, are associated with increased citations, media attention, collaborations, and funding opportunities, providing significant career benefits to researchers.
McKiernan_2017_eLife_How open science helps researchers succeed.pdf
Nosek, B. A., Ebersole, C. R., DeHaven, A. C., & Mellor, D. T. (2018). The preregistration revolution. Proceedings of the National Academy of Sciences, 115(11), 2600-2606. https://doi.org/10.1073/pnas.1708274114
Discusses how preregistration distinguishes hypothesis generation from hypothesis testing, addresses cognitive biases like hindsight bias, and improves the credibility of research findings by specifying analysis plans before observing results.
Scheel, A. M., Schijen, M. R. M. J., & Lakens, D. (2021). An excess of positive results: Comparing the standard psychology literature with Registered Reports. Advances in Methods and Practices in Psychological Science, 4(2), 1-12. https://doi.org/10.1177/25152459211007467
Compares positive result rates between Registered Reports (44%) and standard publications (96%) in psychology, suggesting that the Registered Report format substantially reduces publication bias and/or Type I error inflation in the published literature.
Klonsky, E. D. (2024). Campbell's Law explains the replication crisis: Pre-registration badges are history repeating. Assessment, 32(2), 224-234. https://doi.org/10.1177/10731911241253430
Argues that preregistration mandates, particularly badges, exemplify Campbell's Law by converting a useful tool into an indicator of quality, potentially distorting scientific practice and harming psychological science in unanticipated ways similar to past misuses of hypotheses and p-values.
Klonsky, E. D. (2024). How to produce, identify, and motivate robust psychological science: A roadmap and a response to Vize et al. Assessment, 32(2), 244-252. https://doi.org/10.1177/10731911241299723
Presents a three-part vision for cultivating robust psychological science: knowing how to produce robust findings (sufficient samples, valid measurement, transparency), identifying robust findings through four types of robustness checks, and marshaling sociocultural forces to reward rigorous research rather than cosmetic indicators of rigor.
Nuzzo, R. (2015). How scientists fool themselves—and how they can stop. Nature, 526, 182-185. https://doi.org/10.1038/526182a
Explores how human cognitive biases and self-deception contribute to the reproducibility crisis in science, discussing how researchers' natural tendency to find patterns and accept reasonable-seeming results can lead them astray without realizing it.
Rubin, M. (2020). Does preregistration improve the credibility of research findings? The Quantitative Methods for Psychology, 16(4), 376-390. https://doi.org/10.20982/tqmp.16.4.p376
Provides a critical review of 17 proposed benefits of preregistration, distinguishing between historical and contemporary transparency, and concludes that preregistration's historical transparency does not facilitate credibility judgments beyond what contemporary transparency (clear rationales, public data, robustness demonstrations) provides.
Rubin_2020_Does preregistration improve the credibility of research findings.pdf
Sarafoglou, A., Kovacs, M., Bakos, B., Wagenmakers, E.-J., & Aczel, B. (2022). A survey on how preregistration affects the research workflow: Better science but more work. Royal Society Open Science, 9, 211997. https://doi.org/10.1098/rsos.211997
Surveys 355 researchers (299 with preregistration experience) about perceived benefits and challenges of preregistration, finding that researchers believe it improves research quality but increases work-related stress and project duration, with experienced users more positive than non-users.
Sarafoglou_2022_A survey on how preregistration affects the research workflow.pdf
Soderberg, C. K., Errington, T. M., Schiavone, S. R., Bottesini, J., Thorn, F. S., Vazire, S., Esterling, K. M., & Nosek, B. A. (2021). Initial evidence of research quality of registered reports compared to the standard publishing model. [Preprint]
Reports that 353 researchers rated 29 Registered Reports higher than 57 matched non-RR papers across all 19 quality criteria (mean difference=0.46), with largest improvements in methodological rigor (0.99) and analysis (0.97), and smallest in novelty (0.13) and creativity (0.22).
Vize, C. E., Phillips, N. L., Miller, J. D., & Lynam, D. R. (2024). On the use and misuses of preregistration: A reply to Klonsky (2024). Assessment, 32(2), 235-243. https://doi.org/10.1177/10731911241275256
Responds to Klonsky's critique by providing conceptual clarification on preregistration's purpose, distinguishing legitimate concerns about preregistration badges from misguided concerns about preregistration practice itself, and outlining challenges and opportunities for effective implementation.
Vize_2024_On the uses and misuses of preregistration - A reply to Klonsky (2024).pdf
</aside>
<aside> <img src="/icons/feed_orange.svg" alt="/icons/feed_orange.svg" width="40px" />
Preregistration for Qualitative Research Template.docx
Preregistration of Secondary Data Analysis Template.docx
Preregistration of Systematic Reviews.docx
Preregistration Planning and Deviation Documentation (PPDD).docx
| --- | --- | --- | --- |
</aside>
<aside> <img src="/icons/add_gray.svg" alt="/icons/add_gray.svg" width="40px" />
</aside>