9 Preregistration Check

9.1 What it checks

The prereg_check module finds links to preregistrations on the OSF and AsPredicted in the manuscript, retrieves the preregistration text, and organises its contents into a single standardised template. This makes it much easier for a reviewer to compare what was preregistered against what the manuscript reports.

Note

This module makes live network calls to the OSF and AsPredicted to retrieve preregistration content. You need an internet connection to run the code below. (When this book is built, these calls run against the live services.)

9.2 Running the module

demopaper() links to two preregistrations, so it is a good example.

paper <- demopaper()
mo <- module_run(paper, "prereg_check")
mo$traffic_light

#> [1] "info"

mo$summary_text

#> [1] "We found 2 preregistrations."

The table has one row per preregistration, with the template fields organised into columns. Here are a few of them:

mo$table[, c("template_name", "title", "id", "sample_size")] |>
  knitr::kable()

template_name	title	id	sample_size
OSF Preregistration	Papercheck Test	48ncu	We randomly assigned 50 scientists to a condition where their manuscript was automatically checked for errors, and 50 scientists to a control condition with a checklist.
AsPredicted	To err is human	by8i8v	We randomly assigned 50 scientists to a condition where their manuscript was automatically checked for errors, and 50 scientists to a control condition with a checklist.

The full set of fields the module standardises includes the study design, hypotheses, sample-size rationale, stopping rule, variables, statistical tests, inference criteria, and exclusion criteria:

names(mo$table)

#>  [1] "template_name"                "title"                       
#>  [3] "id"                           "link"                        
#>  [5] "date_created"                 "date_modified"               
#>  [7] "date_registered"              "embargo_end_date"            
#>  [9] "ia_url"                       "description"                 
#> [11] "study_type"                   "blinding"                    
#> [13] "study_design_overview"        "data_collection_started"     
#> [15] "existing_data_explanation"    "data_collection_procedures"  
#> [17] "sample_size"                  "sample_size_rationale"       
#> [19] "stopping_rule"                "design_independent_variables"
#> [21] "design_dependent_variables"   "indices"                     
#> [23] "statistical_tests"            "inference_criteria"          
#> [25] "data_exclusion_criteria"      "outliers_and_exclusions"     
#> [27] "exploratory_analyses"         "additional_comments"         
#> [29] "research_questions"           "paper_id"

9.3 Reading a single preregistration’s contents

Because each preregistration becomes a row, you can inspect one in detail by transposing it:

prereg <- mo$table[1, c("template_name", "sample_size", "statistical_tests", "data_exclusion_criteria")]
t(prereg) |>
  knitr::kable()

template_name	OSF Preregistration
sample_size	We randomly assigned 50 scientists to a condition where their manuscript was automatically checked for errors, and 50 scientists to a control condition with a checklist.
statistical_tests	We will perform an independent t-test on the number of mistakes in each group.
data_exclusion_criteria	No exclusions are expected.

9.4 Options and limitations

prereg_check takes only the paper argument. It cannot extract information from:

unstructured preregistration templates (where everything is in one free-text field), or
preregistrations stored as uploaded text documents on the OSF rather than in the structured form.

For the details of which OSF registration templates exist and how they are identified, see the OSF Registration Schemas chapter.

9.5 Notes

If you want Metacheck to support additional preregistration platforms, reach out to the development team.