Submit a Manuscript to the Journal
Computer Science Education
For a Special Issue on
Demonstrating Community-Sourced Replication and Meta-Analysis
Manuscript deadline
Special Issue Editor(s)
Briana Morrison,
University of Virginia, USA
[email protected]
Andrew Petersen,
University of Toronto, Canada
[email protected]
Rachel Wong,
University of Tennessee, Knoxville, USA
[email protected]
Joshua Rosenberg,
University of Tennessee, Knoxville, USA
[email protected]
Demonstrating Community-Sourced Replication and Meta-Analysis
Replication provides one valuable means of increasing confidence in research findings and supporting the development of a cumulative body of knowledge. Despite recent efforts to encourage replication, relatively few replication studies or meta-analyses have been published in computing education, and researchers in the community have noted systematic issues with reporting that make comparison of results difficult. Addressing this challenge will require the engagement of the broader community of computing educators to systematically engage in the replication of published results.
To motivate such contributions and to provide an entry point for educators and researchers from outside computing education to engage with the field, we are pleased to announce Demonstrating Community-Sourced Replication and Meta-analysis, a special issue of Computer Science Education. The issue seeks community engagement to replicate a recently published article, A Randomized Controlled Trial on the Nomenclature of Scientific Computing, and will demonstrate protocols for soliciting and reviewing replication studies and the meta-analyses enabled by such replication studies. The process is modeled after the Bootstrapping in CS Education project (https://depts.washington.edu/bootstrp/), in which a shared study is conducted across institutions, with analyses performed both locally and on aggregated data. This model allows those who are new to computing education research an opportunity to participate in a well-designed research study and learn from experienced researchers in the field.
This special issue is soliciting replications of the published article, A Randomized Controlled Trial on the Nomenclature of Scientific Computing. To facilitate this process, a replication package is provided, consisting of a customizable survey in an easily deployable format (a Docker container [1]) and instructions for modifying and deploying the survey, running analyses, and reporting results. (Some technical support is available if issues are encountered during setup.) Authors will submit a paper for publication that describes their context, data collection methods and deviations from the protocol, an analysis of the results, and a discussion of differences observed. These papers will be peer-reviewed and accepted if they replicate the original paper methodology and conduct data collection and analysis in a methodologically sound manner. Accepted authors will be invited to join the meta-analysis team that will produce an analysis of all accepted works to close the special issue.
This effort to community-source replications and collaboratively analyze the results is new in our field. We welcome contributions from established members of the community as well as new researchers (Masters and early Ph.D. students) and new members of the computing education research community.
If you have questions about this call, please reach out to any of the special issue editors.
[1] a lightweight, standalone, executable package of software that includes everything needed to run an application
Submission Instructions
Submissions to this special issue must be replications of A Randomized Controlled Trial on the Nomenclature of Scientific Computing that (a) execute the original data collection and analysis protocol with fidelity, (b) deploy a similar-enough survey to provide data that is comparable to the original and other replication efforts, (c) provide the raw data for meta-analysis, and (d) describe their context and the analysis with sufficient detail to allow for interpretation of observed differences. Please note, unlike a conventional manuscript submission, submission to this special issue will not need to include an extended introduction/motivation, literature review, or discussion section. Rather, submissions focus on the replication effort (e.g., methods, context, and results) with the full, interpretive discussion deferred to the meta-review that will be collaboratively authored by all scholars that participate in the special issue.
We anticipate that authors may wish to modify or add to the survey items, and some modification is encouraged. We also anticipate that authors in some jurisdictions may be unable to collect all of the data, and we will work with authors when modifications are necessary. To begin the process, authors must complete this expression of interest that describes their context and proposed modifications to the protocol by October 16, 2026. Further detail on the publication process, including submission guidelines and format, will be sent to authors who express interest. To allow for discussion of any modifications and to allow time for ethics approval and data collection, earlier submission of the expression of interest is recommended.
In addition, we understand that authors in some jurisdictions may find it difficult to obtain permission to publish their data as required for this special issue. We welcome queries from potential authors who may need to provide the data directly to the meta-analysis team or who need to limit the data published publicly to satisfy their review board or privacy office.
Key Dates
Mid-June - October 16, 2026: Submission of expression of interest. Data collection can begin anytime after approval.
August-December 2026: Drop-in sessions with editors to discuss the expression of interest or data collection process.
March 31, 2027: Deadline for submission of replication paper, consisting of a description of the context, methods, results, and discussion, and the data set for meta-analysis.
June 1, 2027: Tentative deadline for decisions to be returned to authors. Conditionally accepted and accepted authors will be invited to join the meta-analysis team.
July 15, 2027: Deadline for submission of camera-ready replication papers.
July 30, 2027: Deadline for submission of meta-analysis report.
Publication is tentatively expected in Issue 4 of 2027.