21  Repository Check

21.1 What it checks

The repo_check module finds links to research repositories in a paper — the Open Science Framework (OSF), GitHub, ResearchBox, PsychArchives, and Zenodo — retrieves the list of files in each, and reports what is shared. It is the foundation for every other module that reads repository files: code_check, data_check, and data_validate all build on the file list repo_check produces.

Beyond listing files, repo_check runs a set of checks on what it finds:

• whether each repository has a README;
• whether files are bundled into archives rather than shared individually;
• whether any files are unclassifiable by name or extension;
• whether file names follow machine-parseable conventions;
• whether the studies the manuscript names match how the repository’s files are grouped;
• whether proprietary E-Prime binary files (.edat/.emrg) have a matching plain-text export;
• whether any files sit behind restricted or embargoed access; and
• whether an OSF registration points at a closed or inaccessible source project.

Note

This module makes live network calls to the linked repositories. You need an internet connection to run the code below.

21.2 Running the module

demopaper() links to several repositories.

paper <- demopaper()
mo <- module_run(paper, "repo_check")
mo$traffic_light

#> [1] "yellow"

cat(mo$summary_text)

#> 
#> -  We found 7 files in 1 repository.
#> -  We found 0 README files and 1 repository without READMEs.

The table lists every file found across all linked repositories:

mo$table[, c("repo_name", "file_name", "file_type", "file_size")] |>
  head(10) |>
  knitr::kable()

repo_name	file_name	file_type	file_size
4377	Code/Study 1.r	code	183
4377	Codebook/Study 1 - csv - dataset 1.csv___CODEBOOK.csv	data	96
4377	Codebook/Study 1.csv___CODEBOOK.csv	data	96
4377	Data/Study 1 - csv - dataset 1.csv	data	185
4377	Data/Study 1.csv	data	185
4377	Materials/Study 1 - test txt file.txt	text	12
4377	Table of Contents Box #4377.csv	data	573

21.3 Running on many papers

mo <- module_run(psychsci[1:10], "repo_check")
mo$summary_table

21.4 Repositories it searches

repo_check looks for links to five repository platforms, each via its own link-finding helper (osf_links(), github_links(), rbox_links(), psycharchives_links(), zenodo_links()) and its own file-listing call:

Platform	Link finder	How files are listed
OSF	osf_links()	recursive osf_info(), including registrations (see below)
GitHub	github_links()	github_tree_files() — one repo-metadata + one full-tree API call, gated for size
ResearchBox	rbox_links()	rbox_file_download() (downloads and unzips the box)
PsychArchives	psycharchives_links()	psycharchives_file_download() (DSpace REST API; lists without downloading)
Zenodo	zenodo_links()	zenodo_info()

21.4.1 OSF registrations follow their source project

An OSF registration is a locked snapshot. Many “Open-Ended Registration” entries (common in older schemas) only describe what was shared, with the real files living in the project the registration was made from (registered_from). Since the manuscript usually links only the registration, repo_check follows registered_from automatically:

• if the source project is public, its files are listed under the source project’s own URL (not the registration’s);
• if the source project is closed or inaccessible, the report flags this explicitly, distinguishing two cases: the registration itself still holds a mirrored copy of the files (nothing is lost, but the permanent home is unreachable), or no copy exists anywhere (the files are genuinely unreachable).

21.4.2 GitHub size gating

A GitHub repository can be arbitrarily large, so repo_check gates recursive listing by default:

module_run(paper, "repo_check",
           github_gate = TRUE,               # default
           github_max_repo_size_mb = 500,     # default
           github_max_files = 1000)           # default

A repository over either threshold is skipped (not listed), with a warning naming the repository and the reason. Set github_gate = FALSE to force a full recursive listing regardless of size. Repositories skipped this way are recorded in $gated_repos (see below), so downstream modules and reports can explain why a paper produced no GitHub files, instead of just reporting none found.

21.5 Study grouping

repo_check runs a preliminary classification and grouping pass over every file it finds:

• data_classify_files() labels each file’s data_type (data, code, readme, archive, documentation, output, …) from its name/path/extension;
• .data_doc_role() further labels documentation-type files (readme, license, codebook, …);
• data_group_llm() assigns each file to a study group, using deterministic path/repo-split rules first and falling back to an LLM (llm_use(TRUE)) only when those are ambiguous — see the LLMs chapter. Root-level files that are collection-wide by nature (a top-level README, ro-crate-metadata.json, or LICENSE) are excluded from grouping, since they don’t belong to any one study.

This pass is preliminary: repo_check never downloads file contents (except where a platform’s own listing call requires it, like ResearchBox), so it classifies from names and paths only. data_check reclassifies afterwards using file contents too, and its classification is authoritative for where a file is placed in a paper’s structure. repo_check’s own pass exists to power its own report — the file-naming check, the classification dropdown, and the study/repository mismatch check below.

21.5.1 Study/repository mismatch

If the manuscript explicitly names studies (e.g. “Study 1”, “Study 2”), repo_check compares that roster against the studies the files actually separate into. A mismatch — a named study with no matching files, or a file-derived group not named in the manuscript — is flagged in the report. A manuscript that never names studies explicitly has no roster to compare against, so this check is silently skipped rather than treated as a mismatch.

21.6 File naming conventions

check_file_naming() checks every file name for machine-parseability: no spaces or special characters, and every file classifiable by name or extension (data_type != "unknown"). Issues are split into two severities:

• bad — a real naming problem (should be fixed);
• suggestion — not required, but a good habit (e.g. a more descriptive name).

mo$naming_issues |> head(5) |> knitr::kable()

21.7 Special file types

21.7.1 Proprietary E-Prime files

E-Prime’s .edat/.edat2 (and .emrg/.emrg2 merge files) are proprietary binary formats that only the E-Prime software can open — metacheck neither downloads nor reads them. The analysable data lives in E-Prime’s plain-text export instead. repo_check flags any .edat/.emrg file that has no matching .txt export alongside it (matched by file stem), so authors know to also share the readable export (File → Export in E-Prime).

21.7.2 Restricted-access files

A PsychArchives item flagged restrictedAccess/embargoedAccess hides its files behind institutional (SSO) login — they cannot be fetched programmatically. repo_check flags any repository with restricted files, since metacheck cannot examine them and other researchers cannot reuse them without requesting access.

21.7.3 Non-.zip archives

Only .zip stores its file listing in a way that can be inspected via an HTTP range request without downloading the whole archive; .7z/.rar/.tar.gz must be downloaded whole, and some formats metacheck cannot read at all. repo_check only lists archive files — it never opens one (peeking inside a .zip, or downloading an archive, is data_check’s job) — but it does warn when an archive is in a non-.zip format, since that limits how discoverable and reusable its contents are.

21.8 A clean example and one with problems

The demo paper links to three repositories, and running repo_check on it produces a yellow light — there are problems to address. The summary text reports exactly what:

mo$traffic_light

#> [1] "yellow"

cat(mo$summary_text)

#> 
#> -  We found 7 files in 1 repository.
#> -  We found 0 README files and 1 repository without READMEs.

Here the issues are a missing README and an archive file whose contents could not be examined. The per-paper summary table counts each file type — note files_readme is 0 (no README was shared) and files_zip is 1 (one archive):

mo$summary_table |>
  knitr::kable()

paper_id	repo_n	files_n	files_data	files_code	files_readme	files_zip
to_err_is_human	1	7	5	1	0	0

We can see the specific archive file that was flagged in the full file table:

mo$table[mo$table$file_type %in% "archive", c("repo_name", "file_name")] |>
  knitr::kable()

repo_name	file_name

A clean result, by contrast, would include a README in every linked repository (files_readme ≥ 1 for each), share files individually rather than as a zip (files_zip of 0), have no unclassifiable or badly-named files, agree with the manuscript’s study roster, and have no closed registrations — producing a green light with no items to address.

21.9 Options

repo_check accepts arguments for working with local files instead of (or in addition to) online repositories:

# also include files from a local folder
module_run(paper, "repo_check", local_path = "path/to/downloaded/files")

# only check a local folder, skip all online lookups
module_run(paper, "repo_check", local_path = "path/to/files", local_only = TRUE)

local_only = TRUE is useful when you cannot or do not want to contact external services — for example when checking a reviewer submission you downloaded as a zip. See the Local Files chapter for details.

model and params are passed through to data_group_llm() for study grouping, and only take effect when llm_use(TRUE) and the deterministic grouping rules cannot place a file:

module_run(paper, "repo_check", model = "gpt-4o-mini", params = list(temperature = 0))

21.10 The traffic light and what you get back

Light	Meaning
green	repositories were found; every one has a README; no archive files, unclassifiable files, or bad file names; no study/repository mismatch; and no closed OSF registrations
yellow	at least one repository is missing a README, an archive file was found, files could not be classified, real naming problems were found, the manuscript’s study roster disagrees with the repository’s file grouping, or a registration points at a closed source project
na	no repository links were found in the paper at all

Element	What it contains
$table	one row per file across all repositories: repo_url, file_name, file_path, file_url, file_location, file_size, file_type, data_type, doc_role, group, paper_id
$summary_table	per-paper aggregates: repo_n, files_n, files_data, files_code, files_readme, files_zip, files_unknown, naming_issues, roster_mismatch
$naming_issues	full output of check_file_naming() — one row per issue, with severity
$roster_check	the manuscript-vs-file study roster comparison (roster, missing, extra), or NULL if not applicable
$gated_repos	repositories found but not listed (e.g. a GitHub repo over the size gate, or a private OSF component), with the reason
$group_no_evidence	TRUE if study grouping had no evidence to work from
$report	formatted report sections: repositories, files, README, archives, E-Prime, restricted access, closed registrations, unclassified files, roster mismatch, naming, and a full classification breakdown
$summary_text	plain-text bulleted summary of each check

21.11 Related

The companion code_check module analyses the actual code files that repo_check discovers. data_check re-downloads and reclassifies the data files using their contents (not just names/paths), and data_validate runs quality checks on the columns it finds.