Preregistration

This article is licensed under CC0 for maximum reuse.

Preregistration is the practice of documenting your research plan at the beginning of your study and storing that plan in a read-only public repository such as OSF Registries or the National Library of Medicine's Clinical Trials Registry.

Benefits of Preregistration

The benefits of preregistration depend directly on what kind of information, and how much of it, is included inside that preregistration.

A very sparse outline of a study plan may be sufficient to increase the discoverability of the research and thus help to address the file drawer effect (Rosenthal, 1979; Franco, Malhotra, & Simonovits, 2014), yet insufficient to assist in evaluating claims resulting from that research. Including a detailed analysis plan in the preregistration may additionally help reduce unintentional false positive inflation of results (Forstmeier, Wagenmakers, & Parker, 2017) and better enable readers to distinguish exploratory from hypothesis-testing elements in a study (Nosek, Ebersole, DeHaven, & Mellor, 2018). Both modes of research are essential for science to advance, but presenting the results of data-dependent, exploratory discoveries using the tools of statistical inference designed for confirmatory studies makes the results appear more surprising, and publishable, which comes at the expense of their credibility (Nosek, Spies, & Motyl, 2012).

While we recommend preregistering all types of research, the most benefits accrue when performing hypothesis testing, confirmatory, research and these benefits are of particular importance to addressing issues of reproducibility in the published literature (Munafò et al., 2017). Our strongest recommendation is therefore to preregister confirmatory research and to include a detailed analysis plan in that preregistration. If setting out on purely exploratory research or pilot studies, preregistration can still help you remember that intention at the end of that project, improve the transparency of your research

Creating a Preregistration on OSF

OSF welcomes preregistrations from any field or discipline and we support researchers using OSF as a preregistration platform whether or not they will also be using its data storage, collaboration management, or publishing capabilities. Details on how to create a preregistration on OSF are covered in the " Create A Preregistration" guide.

Reporting the results of preregistered research

When reporting the results of preregistered research, there are four basic rules to follow:

Include a link to the original preregistration. This will allow readers to compare the planned and reported study.
Report the results of all pre-specified work. If you set out to run 20 tests, report all of their results. This is especially urgent if only a few of those tests were statistically significant. Since some will be significant by chance alone, readers deserve the full context of the results in order to interpret the overall findings.
Clearly label any unplanned analyses. Plans change (see "Preregistration: A Plan, Not A Prison"). Diverging from your preregistered study plan does not invalidate that preregistration. If changes from the original plan are made, they should simply be transparently documented in the resulting research. This transparency enables readers to better evaluate any impact changes from an original plan may have on the results (Simmons, Nelson, & Simonsohn, 2011).
Finally, include a “Transparent Changes” document for any deviations that occured from your preregistered plan. This may be unnecessary if your exploratory analysis section covers all differences, but in many studies there will be additional changes. Perhaps participant recruitment was tougher than imagined. Or perhaps some measures were impossible to collect. Whatever the changes are, document them and include them in a footnote or separate section.

For additional materials discussing what information should be included in a preregistration, how preregistration can be effectively implemented, and examples of selected preregistrations from multiple disciplines, refer to the Center for Open Science Preregistration Page.

Resources

Center for Open Science Preregistration information page (containing a much more exhaustive list of resources) - Https://Cos.Io/Prereg/
"Preregistration: A Plan, Not a Prison" - Https://Cos.Io/Blog/Preregistration-Plan-Not-Prison/

References

Rosenthal, R. (1979). The file drawer problem and tolerance for null results. Psychological Bulletin, 86(3), 638–641. Https://Doi.Org/10.1037/0033-2909.86.3.638
Franco, A., Malhotra, N., & Simonovits, G. (2014). Publication bias in the social sciences: Unlocking the file drawer. Science, 345(6203), 1502–1505. Https://Doi.Org/10.1126/Science.1255484
Forstmeier, W., Wagenmakers, E.-J., & Parker, T. H. (2017). Detecting and avoiding likely false-positive findings – a practical guide. Biological Reviews, 92(4), 1941–1968. Https://Doi.Org/10.1111/Brv.12315
Nosek, Brian A., Ebersole, C. R., DeHaven, A. C., & Mellor, D. T. (2018). The preregistration revolution. Proceedings of the National Academy of Sciences, 115(11), 2600–2606. Https://Doi.Org/10.1073/Pnas.1708274114
Nosek, B. A., Spies, J. R., & Motyl, M. (2012). Scientific Utopia: II. Restructuring Incentives and Practices to Promote Truth Over Publishability. Perspectives on Psychological Science, 7(6), 615–631. Https://Doi.Org/10.1177/1745691612459058
Munafò, M. R., Nosek, B. A., Bishop, D. V. M., Button, K. S., Chambers, C. D., Percie du Sert, N., … Ioannidis, J. P. A. (2017). A manifesto for reproducible science. Nature Human Behaviour, 1(1), 0021. Https://Doi.Org/10.1038/S41562-016-0021
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant. Psychological Science, 22(11), 1359–1366. Https://Doi.Org/10.1177/0956797611417632

This article is licensed under CC0 for maximum reuse.