Journal of Research on Educational Effectiveness
JREE Announces New TOP Disclosure Practices
JREE movement in the scientific ecosystem toward empirical research that is more transparent, open, and reproducible. Consistent with JREE’s values, it is our view that open science facilitates the alignment of scientific practice with scientific ideals, accelerates scientific discovery, broadens access to scientific knowledge, and improves the credibility of empirical research on the effectiveness of educational interventions. To operationalize this support of open science, JREE is a signatory of the Transparency and Openness Promotion (TOP) Guidelines: a set of modular standards for moving published empirical research toward greater openness.
As of April 1, 2022, JREE will be a “Level 1” implementer of all standards in the TOP Guidelines. As a “Level 1” implementer, JREE will require authors toco disclose whether or not they have used open science practices in the TOP Guidelines. In other words, while we will not require the open science practices themselves for publication, we will require authors to disclose whether or not studies included the following practices and include this disclosure in all articles:
● Study and Analysis Plan Preregistration: whether or not the study was registered (e.g., in the Registry of Efficacy and Effectiveness Studies) and, if registered, whether the registration was prospective and included an analysis plan.
● Data, Code, and Materials Transparency: whether or not the data, code, and other materials (e.g., questionnaires, surveys, instructions) used to conduct the research are publicly available—and, if available, where and how to access them.
● Citation Standards: authors should cite all data, code, and other materials in-text and in the references section, if possible.
● Design and Analysis Reporting Guidelines: for randomized trials and systematic reviews, authors will be encouraged to develop their manuscripts in adherence with py checklists developed by the Editors (based on CONSORT-SPI and PRISMA, respectively).
● Publication Bias: authors will be asked to confirm that their manuscript is an honest, accurate, and transparent account of the study being reported.
● Replication: authors will be asked to indicate whether their study is a replication of a previous study—and, if so, provide a citation to that study.
To implement these policies, all manuscripts that move beyond initial review will be asked to complete a “TOP Guidelines Disclosure Form” upon resubmission. This form will ask for details about each of the above practices, which will then inform disclosure statements in all published articles, as well as the awarding of Open Science Badges for qualifying articles. In addition, JREE explicitly welcomes replication studies, as well as papers with strong methods that find null or negative results.
Any questions or feedback about these upcoming changes and drafts of associated forms are most welcome to Methodological Transparency Editor, Sean Grant, [email protected].