32 Replication Check

32.1 What it checks

The ref_replication module cross-references the paper’s cited references against the FLoRA replication database. If you cite an original study that has since been the subject of a replication attempt, that is highly relevant context — especially if the replication failed. The module flags cited originals that have known replications (or reproductions), and can report the replication’s outcome. A replication that the paper already cites is not flagged.

Note

This module runs offline: it matches references by DOI against a copy of the FLoRA database that ships with Metacheck, so the code below needs no internet connection. Only updating that copy with FLoRA_update() (see Updating the database) downloads data.

32.2 Running the module

Without R

This module also runs on Metacheck’s website: upload a paper under Run Metacheck on a Paper, and tick it under Checks (it is not one of the default checks). The Docker installation offers the same website on your own computer. The website runs the module with its default settings; to change them, run it in R as below.

demopaper() cites an original study that has a recorded replication.

paper <- demopaper()
mo <- module_run(paper, "ref_replication")
mo$traffic_light

#> [1] "info"

mo$summary_text

#> [1] "We found 1 replication for 1 original you cited."

The table matches each cited original to its replication(s):

mo$table[, c("doi", "replication_doi", "replication_url")] |>
  knitr::kable()

doi	replication_doi	replication_url
10.1037/0003-066x.54.6.408	10.1177/1474704919852921	NA

32.3 Option: show replication outcomes

By default the module reports that a replication exists. Set show_outcomes = TRUE to also include the outcome of each replication (for example, whether it was successful), which adds the replication_outcome and replication_type information to the report:

mo <- module_run(paper, "ref_replication", show_outcomes = TRUE)
mo$table[, c("doi", "replication_doi", "replication_outcome", "replication_type")] |>
  knitr::kable()

doi	replication_doi	replication_outcome	replication_type
10.1037/0003-066x.54.6.408	10.1177/1474704919852921	failed	replication

32.4 Running on many papers

mo <- module_run(psychsci[1:10], "ref_replication", show_outcomes = TRUE)
mo$summary_table

32.5 Interpreting the result

A match means a replication of a cited study exists in the database — it is an invitation to check whether the replication’s result changes how you should cite the original. As always, the module surfaces information for human judgement.

32.6 Options

Argument	Default	Effect
`paper`	—	the paper or paperlist to check
`show_outcomes`	`FALSE`	include the outcome and type of each replication

32.7 Updating the database

The module matches references against a data frame that ships with Metacheck, which the function FLoRA() returns. The FLoRA replication database grows as new replications are recorded, so the bundled copy gradually goes out of date. Two functions help you manage it:

FLoRA_date()     # the date the bundled database was last updated
FLoRA_update()   # download the newest version of the FLoRA database

FLoRA_update() downloads the latest FLoRA data (>5 MB), summarises it into a compact form, deletes the large original, and caches the result so subsequent checks use the fresh data. Run it occasionally to pick up replications added since your installed version of Metacheck.